Time-of-flight camera for imaging through optical cavity

ABSTRACT

In illustrative implementations, an imaging system may comprise a lens, an optical cavity and a time-of-flight camera. The imaging system may capture an image of a scene. The image may be formed by light that is from the scene and that passes through the optical cavity and the lens. In some cases, the lens is in front of the optical cavity, enabling the Euclidean distance between the lens and the camera sensor to be less than the nominal focal length of the lens. In some cases, the lens is inside the optical cavity, enabling the camera to acquire ultrafast multi-zoom images without moving or changing the shape of any optical element. In some cases, the lens is behind the optical cavity, enabling the system to perform ultrafast multi-spectral imaging. In other cases, an optical cavity between the scene and time-of-camera enables ultrafast ellipsometry measurements or ultrafast spatial frequency filtering.

RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional Application No.62/407,927, filed Oct. 13, 2016 (the “Provisional Application”), theentire disclosure of which is herein incorporated by reference.

FIELD OF TECHNOLOGY

The present invention relates generally to time-of-flight imaging.

SUMMARY

In illustrative implementation of this invention, an imaging system maycomprise a lens, an optical cavity and a time-of-flight (“ToF”) orultrafast sensor. The imaging system may capture an image of a scene.The image may be formed by light that is from the scene and that passesthrough the optical cavity and the lens. In some cases, the lens is infront of the optical cavity, enabling the Euclidean distance between thelens and the camera sensor to be less than the nominal focal length ofthe lens. In some cases, the lens is inside the optical cavity, enablingthe camera to acquire ultrafast multi-zoom images without moving orchanging the shape of any optical element. In some cases, the lens isbehind the optical cavity, enabling the system to perform ultrafastmulti-spectral imaging. In other cases, an optical cavity between thescene and time-of-flight camera enables ultrafast ellipsometrymeasurements or ultrafast spatial frequency filtering.

In some implementations, an optical cavity (“OC”) is located in anoptical path between (i) a scene and (ii) a ToF or ultrafast camera thatis capturing an image of the scene. One or more walls of the cavity maycomprise semi-transparent mirrors. Light from the scene may enter theoptical cavity by passing through a semi-transparent mirror (“STM”) ofthe cavity, and may then reflect repeatedly inside the cavity. Whenlight inside the cavity strikes an STM wall of the cavity, a portion ofthe light may be reflected back into the cavity and a portion of thelight may be transmitted through the STM (and thereby escape thecavity). Thus, during reflections that occur at the end of the firstpass of light in the cavity and at the end of each roundtrip of light inthe cavity, respectively, a portion of the light may be transmittedthrough a STM and thereby escape the cavity. This escaped light may thentravel to the ToF camera.

In some use scenarios: (a) a pulse of light from the scene enters thecavity; and (b) the light that escapes the cavity (after the first passand after each roundtrip inside the cavity) and travels to the ToFcamera comprises a temporal sequence of light pulses (“exit pulses”),such that an exit pulse occurs at the end of the first pass and at theend of each roundtrip, respectively.

In illustrative implementations of this invention, the temporalresolution of the ToF camera is faster than the temporal duration of aroundtrip of light inside the cavity. For example, in someimplementations: (a) a pulse of light from the scene enters the opticalcavity; (b) exit pulses of light exit the cavity; and (c) the ToF camerahas sufficient temporal resolution to distinguish each exit pulse.

If conventional imaging techniques were employed (e.g., a conventionalslow camera), then placing an optical cavity in the optical path betweenthe scene and the camera would be a disaster for imaging, because thecavity would tend to greatly degrade the light signal from the scene bysuppressing non-resonant frequencies.

However, in illustrative implementations of this invention, this problemis solved. In illustrative implementations, the ToF camera acquires dataabout light exiting the optical cavity during an extremely short periodof time before the effects of optical resonance in the cavity becomepronounced. This short period may begin at a time when light from thescene enters the cavity and may end when a relatively small number ofroundtrips of that light in the cavity have occurred (e.g., 100 or lessroundtrips). Or this short period may begin at a time when light fromthe scene enters the cavity and may end when the integral power of thedominant mode of that light in the optical cavity is equal tohalf-maximum of the integral power of the evolving wavefront of thatlight inside the optical cavity. In illustrative implementations, duringthis short period of time, the effects of optical resonance do notsignificantly degrade the light signal from the scene. Thus, inillustrative implementations, the light signal from the scene that theToF camera captures is not significantly degraded by optical resonance,even through the light from the scene passes through an optical cavity.

Passing light from the scene through an optical cavity, before itreaches the ToF camera—and before the effects of resonance in the cavitybecome significant—has many practical applications for ultrafastimaging. These practical applications may include: (a) achieving asmaller form factor; (b) ultrafast multi-zoom imaging; (c) ultrafastmulti-spectral imaging; (d) ultrafast ellipsometric imaging, and (e)ultrafast spatial frequency filtering.

In some cases, the optical cavity enables a much smaller form factor fora ToF imaging system, by folding the optical path between a lens and aToF camera sensor. For example, in some cases: (a) an optical cavity islocated between a lens and the ToF camera sensor; and (b) the cavityfolds the optical path between the lens and the camera, thereby reducingthe Euclidean physical distance between the lens and the rear focalplane of the lens. This folding of the optical path (by reflectionsinside the cavity) enables the Euclidean physical distance between alens and a ToF camera sensor to be greatly reduced (e.g., to an order ofmagnitude less than the nominal focal length of the lens). Thus, theoptical cavity may facilitate a much more compact form factor for a ToFimaging system (or may facilitate the use of longer focal lengthlenses).

In some implementations of this invention, the optical cavity enablesultrafast, multi-zoom imaging. For example, in some cases: (a) anoptical cavity is located in front of a ToF camera; (b) a lens islocated inside the optical cavity; (c) light passes through the lens(which is inside the cavity) repeatedly, such as during the first passof light inside the cavity and during each roundtrip of light inside thecavity, respectively; (d) as the light reflecting inside the cavitypasses through the lens repeatedly, the light is further diffractedduring each pass of light through the lens; and (e) each pass of lightthrough the lens causes an optical system (comprising the lens andoptical cavity) to be focused at a different depth of the scene, becauseeach pass of light through the lens changes the convergence ordivergence of the wavefront. Thus, in some cases, a lens inside theoptical cavity enables the ToF camera to function as multi-zoomultrafast camera; one that is configured to focus on two or more focalplanes at different depths (e.g., depths that differ by an order ofmagnitude or more) with different magnification factors. The multi-zoomimage may be captured during a single ultrafast acquisition frame usinga single set of optical elements, without physically altering the focallength of the lens and without physically moving any optical element.

In some implementations, a set of optical cavities (each of which has adifferent cavity size) time-encodes different spectrums from differentspectral filters. In some cases, these time-encoded spectrums from thedifferent spectral filters enable a ToF camera to performsingle-acquisition multi-spectral imaging at ultrafast rates. Forexample, in some cases, these time-encoded spectrums enable the ToF toperform multi-spectral imaging at the nominal temporal resolution of theToF camera, with no loss of spatial resolution and with no loss oftemporal resolution.

For example, in some implementations for multi-spectral imaging, theentrance optics of a ToF imaging system include a uniaxial configurationof filters. For example, the entrance optics may include an ND filterthat transmits a broad band of frequencies and also include multiplenotch filters. The ND filter and notch filters may be arranged in auniaxial configuration, such that the ND filter is in front of a “stack”of notch filters, which are in front of a lens, which is in front of aToF camera. Each notch filter may transmit substantially all light inthe broad band, except that each notch filter may reduce transmission oflight in a particular notch (subband) of frequencies. For example, eachnotch filter may reflect a majority of light in its respective notch andmay transmit a minority of light in its respective notch. The notch(subband) of frequencies may be different for each of the notch filters.The number of optical cavities in the imaging system may be equal to thenumber of notch filters. Each of the optical cavities, respectively, maybe formed by the broadband ND filter and one of the notch filters.Because each notch filter may have a different notch of frequencies, thespectral composition of light that reflects in a complete roundtrip ineach cavity, respectively, may be different. Furthermore, the size ofthe optical cavities may differ, because the distance between thebroadband filter and the cavity's notch filter may differ from cavity tocavity. Thus, the duration of a roundtrip of light in the differentcavities may differ. This, in turn, may cause light to exit thedifferent cavities at different times and arrive at a ToF camera atdifferent times, thereby time-encoding spectral information (becauselight exiting the different optical cavities may have a differentspectral composition, due to the different notches).

Alternatively, in some multi-spectral implementations of this invention,two or more spectral filters are positioned side-by-side (instead ofbeing located in a uniaxial arrangement). For example, in someimplementations: (a) two spectral filters are located side-by-side; (b)there are two optical cavities, one cavity behind each of the spectralfilters, respectively; (c) each optical cavity is formed by two NDfilters; (d) the two optical cavities are in front of a lens, which isin front of a ToF camera; (e) a different spectral composition of lightexits the first cavity than the second cavity; because differentspectral filters are in front of the two optical cavities; (f) the sizeof the optical cavities differ, because the distance between the two NDfilters in the first cavity is different than in the second cavity; and(e) the different size of the optical cavities causes the duration of aroundtrip of light in the cavities to be different. Thus, in the exampledescribed in the previous sentence, the optical cavities: (a) may delaythe first and second spectrums by different amounts of time due to thedifferent roundtrip distances in the cavities; and (b) may therebytime-encode spectral information.

Alternatively, each of the spectral filters may be located inside anoptical cavity: e.g., a first spectral filter may be positioned insidethe first optical cavity and a second spectral filter may be positionedinside the second optical cavity. Placing a spectral filter inside anoptical cavity—such that light reflecting repeatedly inside the cavitypasses through spectral filter repeatedly—may tend to sharpen thespectral window of the filter in an exponential manner, because thelight signal passing through the filter may be multiplied by the filtereach time that light passes through the filter.

In many embodiments, each of these three applications (smaller formfactor, multi-zoom and multi-spectral), may be implemented with compactcoaxial optics that fit into the form factor of a practical conventionalcamera lens.

In some implementations of this invention, the optical cavity enablesultrafast ellipsometry measurements. For example, in some cases: (a) aring optical cavity is located in an optical path between a scene and aToF camera; and (b) a half-wave plate is located inside the ring cavity.The half-wave plate rotates the polarization of light, each time thatlight passes through the half-wave plate (e.g., during each roundtrip oflight inside the optical cavity). After each roundtrip of light in thering cavity, a portion of light exits the cavity by passing through asemi-transparent mirror (STM), then passes through a linear polarizer(which is in a fixed position), and then travels to a ToF camera.Because the polarization of light is rotated by a small angle in eachroundtrip, the polarization of light exiting the STM (and striking thelinear polarizer) is different for different roundtrips. Thus, aftereach roundtrip in the cavity, the fixed linear polarizer filters adifferent portion of the polarization spectrum of the initial light thatentered the cavity. In some implementations, this configuration(half-wave plate inside ring cavity) enables the ToF camera to takeellipsometry measurements.

In some implementations of this invention, the optical cavity enablesultrafast filtering of spatial frequencies of a light signal. Forexample, in some cases: (a) a collimating lens outputs collimated lightthat, while collimated, passes through a Fourier plane and then, whilestill collimated, is incident on the optical cavity; (b) a mask islocated inside the cavity; and (c) the mask filters different spatialfrequencies of light, each time that light passes through the mask(e.g., during roundtrips of light inside the optical cavity). Withoutbeing limited by theory, this may be because: (a) each point in aFourier plane may correspond to a different spatial frequency; and (b)light may strike different points in the mask in different roundtripsinside the cavity (due to the fact that the optical cavity is unstable),and thus different spatial frequencies may be filtered in each roundtripof light in the cavity. A portion of the light inside the optical cavityescapes the cavity at the end of the first pass in the cavity and at theend of each roundtrip in the cavity, respectively, and then travelsthrough an imaging lens and reaches a ToF camera. This configuration(mask inside unstable optical cavity, which in turn is in a Fouriervolume created by a collimating lens) may enable the ToF camera toperform ultrafast filtering of spatial frequencies of a light signal.

In each of the five applications discussed above (smaller form factor,multi-zoom, multi-spectral, ellipsometric and spatial frequencyfiltering): (a) the temporal resolution may still be the nominaltemporal resolution of the ultrafast ToF camera; and (b) thereforeultrafast phenomena may be imaged with no loss of spatial or temporalresolution.

The Summary and Abstract sections and the title of this document: (a) donot limit this invention; (b) are intended only to give a generalintroduction to some illustrative implementations of this invention; (c)do not describe all of the details of this invention; and (d) merelydescribe non-limiting examples of this invention. This invention may beimplemented in many other ways. Likewise, the description of thisinvention in the Field of Technology section is not limiting; instead itidentifies, in a general, non-exclusive manner, a field of technology towhich some implementations of this invention generally relate.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A, 1B and 1C together show an example of hardware of a ToFimaging system.

FIG. 1A shows illumination hardware and a mask.

FIG. 1B shows mirrors that steer light.

FIG. 1C shows entrance optics in front of a ToF camera.

FIG. 2A shows “lens before optical cavity” configuration of opticalelements.

FIG. 2B shows a “lens inside optical cavity” configuration of opticalelements

FIG. 2C shows a “lens behind optical cavity” configuration of opticalelements.

FIG. 3 shows folding of optical paths in an optical cavity.

FIG. 4 shows another example of hardware of a ToF imaging system.

FIG. 5A shows another example of a “lens before optical cavity”configuration of optical elements.

FIG. 5B shows another example of a “lens inside optical cavity”configuration of optical elements.

FIG. 5C shows another example of a “lens behind optical cavity”configuration of optical elements.

FIG. 6A comprises an x-t streak image of a point source of light.

FIG. 6B comprises an x-t streak image of an illuminated mask.

FIG. 6C comprises five different, unfocused, x-y images of the samemask.

FIG. 6D comprises a focused x-y image of the same mask.

FIG. 7 shows a top view of a multi-zoom ToF imaging system.

FIG. 8 shows a top view of a multi-spectral ToF imaging system.

FIG. 9 is a chart of transmission intensity as a function of number ofoptical cavity round trips, for different optical densities of NDfilters.

FIG. 10 shows a ToF imaging system that takes ellipsometry measurements.

FIG. 11 is a diagram that shows an example of a Fourier region and afractional Fourier region of an optical system.

FIG. 12 shows a ToF imaging system that performs ultrafast filtering ofspatial frequencies.

FIG. 13 is a flowchart of an image processing method.

FIG. 14 is a flowchart of a method, in which light passes through a“lens before optical cavity” configuration of optical elements.

FIG. 15 is a flowchart of a method, in which light passes through a“lens inside optical cavity” configuration of optical elements.

FIG. 16 is a flowchart of a method, in which light passes through a“lens after optical cavity” configuration of optical elements.

FIG. 17 shows an example in which spatial divergence of light increasesas the number of cavity roundtrips increases.

FIG. 18A shows a top view of another multi-spectral ToF imaging system.

FIG. 18B and FIG. 18C are charts that show transmission and reflectance,respectively, for this other multi-spectral system.

The above Figures show some illustrative implementations of thisinvention, or provide information that relates to those implementations.The examples shown in the above Figures do not limit this invention.This invention may be implemented in many other ways.

DETAILED DESCRIPTION

FIGS. 1A, 1B and 1C together show an example of hardware of a ToFimaging system, in an illustrative implementation of this invention.

FIG. 1A shows illumination hardware and a mask, in an illustrativeimplementation of this invention. In the example shown in FIG. 1A, theillumination hardware includes a light source 101, mirrors 103, 105, abeam expander 107, and a diffuser 109. In FIG. 1A, the light source 101emits light that is steered by mirror 103, then steered by mirror 105,then spread out by beam expander 107, and then diffused by diffuser 109to create a diffuse backlight.

In many implementations of this invention, the light source 101comprises a pulsed laser. For example, in some implementations, thelight source comprises a Mira® 900 mode-locked Ti-Sapphire laser thathas a short pulse width or a SuperK® supercontinuum laser that producesmulti spectral light.

However, this invention is not limited to lasers and is not limited toany particular kind of light. The light source 101 may comprise anyartificial active light source, including a laser or LED (light emittingdiode). The light source may emit any kind of light, including pulsedlight, light that is not pulsed, coherent light, collimated light,uncollimated light, narrow spectrum light, wide spectrum light, visiblelight, infrared light, or ultraviolet light.

In FIGS. 1A, 1B and 1C: (a) a mask 111 functions as a transmissivescene; and (b) a ToF camera 125 captures one or more images of thistransmissive scene. For example, in some implementations, the scene thatis imaged comprises a mask that is placed behind the diffuser and thatcomprises an opaque polyvinyl sheet with a cutout, or comprises amulti-colored polyvinyl sheet.

However, this invention is not limited to any particular type of scene,which is imaged by the ToF camera. For example, the scene may betransmissive (such that, light is transmitted through the scene beforereaching the ToF camera) or reflective (such that light reflects fromthe scene before reaching the ToF camera), or may be both transmissiveand reflective. The ToF camera may capture images of a scene, where thelight from the scene includes light that reflects from the scene andlight that is transmitted through the scene.

FIG. 1B shows mirrors that steer light, in an illustrativeimplementation of this invention. In the example shown in FIGS. 1A, 1Band 1C, mirror 112 and mirror 117 steer light from a scene, such thatlight from the scene travels to a ToF camera 125. Mirror 112 isrotatable: an actuator 115 may actuate rotation of a rod 114 that isattached to mirror 112, thereby rotating mirror 112. Mirror 112 may berotated in order to scan different portions of the scene being imaged.For example, in some implementations of this invention: (a) the ToFcamera comprises a streak camera that captures x-t images, where eachx-t image measures spatial information in only the horizontal xdimension; (b) mirrors 112 and 117 comprise a periscope mirror set; (c)mirror 112 is rotated to different angular positions, such that the ToFcamera captures an x-t image at different y positions of the scene; and(d) data from the x-t images for all of they positions, respectively,are combined to create an x-y-t data cube. Thus, the x-y-t data cube maycomprise data regarding light that is incident on a 2D surface (with x-ydimensions) at different times during a temporal period.

However, in some cases, the scanning periscope mirrors are omitted. Forexample, in some cases, the ToF camera 125 measures spatial informationin both the x and y dimensions at the same time; and thus, there is noneed for a scanning mirror to scan different regions (e.g., different ypositions) in the scene. For example, one may avoid they scan by using a2D single-photo-avalanche diode array (SPAD) camera or 2D ToF sensor orby using a lenslet array to achieve 2D image acquisition.

FIG. 1C shows entrance optics in front of a ToF camera, in anillustrative implementation of this invention. In the example shown inFIGS. 1A, 1B and 1C, light from the scene is steered by mirrors 112,117, then passes through entrance optics 121, and then is measured by aToF camera 125.

In illustrative implementations, the entrance optics 121 comprises oneor more optical cavities, one or more lenses, and optionally one or moreother optical elements (such as a waveplate, mask, spatial lightattenuator, or other reflective, transmissive or attenuating opticalelement).

The one or more lenses in the entrance optics 121 may comprise a singlelens, compound lens or a lens system. Any type of lens may be employedin the entrance optics 121. For example, in some cases, the entranceoptics 121 includes one or more achromatic doublet lenses.

The one or more optical cavities in the entrance optics 121 may be ofany type. For example, an optical cavity (“OC”) in the entrance optics121 may comprise a Fabry-Perot (“FP”) optical cavity (such as a FP solidoptical etalon, or a FP air-spaced optical etalon), any other type oflinear OC, a ring OC, or a bowtie OC. One or more reflective surfaces ofthe optical cavity may comprise a semi-transparent mirror (“STM”). Insome implementations: (a) each optical cavity in the entrance optics 121is a FP cavity; (b) each FP cavity comprises two reflective, neutraldensity filters (“ND filters”); (c) each of the ND filters is asemi-transparent mirror that, when struck by incident light, reflects aportion of incident light and transmits (through the ND filter) aportion of the incident light; and (d) in each FP cavity, the two NDfilters are approximately parallel to each other, such that light insidethe cavity reflects repeatedly inside the cavity, back and forth betweenthe two ND filters.

The one or more optical cavities in the entrance optics 121 may bestable or unstable. To say that an optical cavity is “unstable” meansthat light reflecting inside the cavity tends to move further andfurther away from an axis of the cavity in each successive reflection,until the light escapes the cavity.

In a prototype of this invention: (a) light from the scene passesthrough entrance optics and then enters a streak camera; (b) theentrance optics comprise a lens and two neutral density reflectivefilters; (c) the streak camera is a Hamamatsu® Orca® R2 C10600 streakcamera with a sweep unit; (d) the streak camera captures x-t images witha spatial x resolution of 672 pixels and a temporal resolution of 512time bins (2 ps each); (e) they dimension is recorded by sweeping aperiscope mirror set; and (f) data from the x-t images at different ypositions is combined to create an x-y-t data cube that may bevisualized. The prototype described in this paragraph is a non-limitingexample of this invention; this invention may be implemented in manyother ways.

In many implementations of this invention, the optical cavity (OC)includes at least one semi-transparent mirror (“STM”). In some usescenarios of this invention, light from the scene may enter the OC, andthen reflect repeatedly inside the OC. During (or at the end of) eachround-trip of light inside the OC), the light may strike a given STM.(For example, the given STM may comprise STM 135 in FIGS. 2A, 2B, 2C, 3,or ND filter 435 in FIG. 4, or ND filter 535 in FIGS. 5A, 5B, 5C, or STM735 in FIG. 7, or STM 835 in FIG. 9, or STM 1033 in FIG. 10, or STM 1235in FIG. 11). This given STM may be partially transmissive and partiallyreflective, such that when light inside the OC is incident on the givenSTM, a portion of the incident light reflects back into the OC andanother portion of the light passes through the given STM and thentravels to a ToF camera. In some implementations: (a) a pulse of lightfrom the scene enters the OC and reflects repeatedly inside the OC, and(b) a temporal sequence of light pulses (which are less intense than thelight pulse that entered the OC from the scene) exit the OC by passingthrough the given STM. These light pulses that exit the OC and travel tothe ToF camera are sometimes called “exit pulses” herein.

In illustrative implementations: (a) the temporal resolution of the ToFcamera is shorter than the duration of a round trip of light in the OC;and (b) the ToF camera distinguishes between light that exits the OC atthe end of different round trips in the OC. For example, in some cases:(a) a pulse of light from the scene may enter the optical cavity; and(b) the temporal resolution of the ToF camera may be sufficiently fastthat the ToF camera detects each individual exit pulse of light thatexits from the OC.

The more times that light reflects back and forth inside the OC, thegreater are the effects of optical resonance in the OC. When the numberof roundtrips of light in the OC becomes sufficiently large, the opticalresonance tends to destroy information contained in non-resonantfrequencies, thereby degrading the light signal from the scene so muchthat it becomes useless for imaging.

In illustrative implementations of this invention, this problem (ofresonance degrading the light signal) is avoided or mitigated, byutilizing data regarding only “early light”. The early light may consistof light from the scene that exits an optical cavity before completingmore than 100 roundtrips inside the cavity. For example, in some casesthe ToF camera may create a digital image, using only data thatrepresents measurements of early light taken by the ToF camera, wherethe early light consists of only light from the scene which entered anoptical cavity and then exited the cavity: (a) after no more than threeroundtrips in the cavity; or (b) after no more than six roundtrips inthe cavity; or (c) after no more than twelve roundtrips in the cavity,or (d) after no more than 100 roundtrips in the cavity.

Alternatively, in some implementations, the early light consists oflight from the scene that exits the optical cavity before the integralpower of the dominant mode of that light in the optical cavity is equalto half-maximum of the integral power of the evolving wavefront of thatlight inside the optical cavity

For example, the ToF camera may capture images formed only by earlylight. Or, data from measurements of only early light may be used togenerate a digital image of the scene, regardless of whether the ToFcamera captures data regarding other light.

For example, in some implementations: (a) a pulse of light from thescene enters an optical cavity; and (b) an initial set of exit pulsesexits the cavity. The initial set of exit pulses may occur before theeffects of resonance become strong, and thus before the light signal issignificantly degraded. Thus, in some implementations: (a) informationin an initial set of exit pulses (e.g., all or a subset of the first 100exit pulses after a pulse of light from the scene enters the OC) isutilized to image the scene; (b) the initial exit pulses occur beforethe effects of OC resonance become strong; and (c) thus, the effects ofOC resonance do not significantly degrade the information about thescene contained in the initial set of exit pulses.

For example, the ToF camera may capture images of only an initial set ofexit pulses. Or, data from only an initial set of exit pulses may beused to image the scene, regardless of whether the ToF camera capturesdata regarding other exit pulses. In some cases, the initial set of exitpulses may comprise all or subset of the first 100 exit pulses that exitthe rear STM after a pulse of light from the scene enters the OC. Forexample, the initial set of exit pulses may comprise: (a) the initialthree exit pulses that exit the rear STM after a pulse of light from thescene enters the OC; (b) the initial six exit pulses that exit the rearSTM after a pulse of light from the scene enters the OC; (c) the initialtwelve exit pulses that exit the rear STM after a pulse of light fromthe scene enters the OC; or (d) the initial 100 exit pulses that exitthe rear STM after a pulse of light from the scene enters the OC. Insome cases, the initial set of exit pulses comprises all or a subset ofthe exit pulses that occur before the integral power of the dominantmode of the OC exceeds half-maximum of the integral power of theevolving wavefront inside the OC.

In the examples shown in FIGS. 2A, 2B and 2C: (a) the entrance optics121 comprise a lens 131, a front semi-transparent mirror (STM) 133, anda rear STM 135; (b) the two STMs together comprise a linear opticalcavity (specifically, an FP optical cavity); and (c) light 130 from thescene (e.g., a pulse of light) enters the entrance optics, passesthrough the entrance optics, and travels to a ToF camera 125.

In some implementations of this invention, the entrance optics 121comprise optical elements that are arranged in a “lens before opticalcavity” configuration, or in a “lens inside optical cavity”configuration, or in a “lens behind optical cavity” configuration.

FIG. 2A shows a “lens before optical cavity” configuration of opticalelements, in an illustrative implementation of this invention. In thisconfiguration, the lens 131 is in front of the optical cavity. Thisconfiguration may (by folding optical paths inside the optical cavity)allow the physical distance between the lens and ToF camera to beshorter than in a conventional ToF imaging system.

In FIGS. 2A, 2B and 2C, the optical cavity comprises a front STM 133 anda rear STM 135.

FIG. 2B shows a “lens inside optical cavity” configuration of opticalelements, in an illustrative implementation of this invention. In thisconfiguration, the lens 131 is inside the optical cavity. Thisconfiguration may be employed for multi-zoom imaging.

FIG. 2C shows a “lens behind optical cavity” configuration of opticalelements, in an illustrative implementation of this invention. In thisconfiguration, the lens is behind the optical cavity. This configurationmay be employed for multi-spectral imaging.

FIG. 3 shows folding of optical paths in an optical cavity, in anillustrative implementation of this invention. This folding of opticalpaths may allow a much smaller form factor for a ToF imaging system.

In the example shown in FIG. 3, a “lens before optical cavity”configuration is used. In this configuration: (a) a converging lens 131is located in front of an optical cavity; and (b) the optical cavity(which is located in optical paths between the lens and the ToF cameraand which comprises STMs 133, 135) folds these optical paths. Thisfolding of optical paths inside the cavity may occur during reflectionsof light inside the cavity, and may allow a ToF camera sensor to beplaced at a distance from lens 131 that is much shorter than the nominalfocal length of lens 131. For example, in FIG. 3: (a) the ToF camerasensor may be located at geometric plane 352, and (b) this geometricplane 352 is located at a distance from lens 131 that is much shorterthan the nominal focal length of lens 131. As used herein, “nominalfocal length” of a lens means the focal length of the lens for lightwhich travels along a straight optical path that starts at a point wherethe light exits the lens and ends at the rear focal plane. In someimplementations, the reduced lens/sensor distance is least an order ofmagnitude less than the nominal focal length of the lens.

In the example shown in FIG. 3: (a) light from a scene passes throughlens 131 (e.g., light rays 301, 302 pass through lens 131); (b) lens 131is plano-convex of biconvex, and thus tends to converge light thatpasses through it; (c) converging light that exits lens 131 enters anoptical cavity by passing through a front STM 133; (d) the opticalcavity comprises front STM 133 and rear STM 135; (e) after entering theoptical cavity, the light reflects repeatedly inside the optical cavity,back and forth between the front and rear STMs; (f) the rear STM 135 ispartially transmissive and partially reflective; (g) each time thatincident light strikes the rear STM 135, a portion of the incident lightreflects back into the cavity and a portion of the incident light passesthrough the rear STM, thereby exiting the rear STM; (h) thus, duringreflections that occur at the end of the first pass of light in thecavity and at the end of each roundtrip of light in the cavity,respectively, a portion of the light may be transmitted through rear STM135 and thereby exit the cavity; (i) this escaped light may then travelto the ToF camera; and (j) if a pulse of light passes through lens 131and enters the cavity, then the light that exits the cavity through therear STM 135 (after the first pass and after each roundtrip inside thecavity) and then travels to the ToF camera may comprise a temporalsequence of light pulses (“exit pulses”), such that an exit pulse occursat the end of the first pass and at the end of each roundtrip,respectively.

As a non-limiting example of terminology used herein: in FIG. 3: (a)after light that comprises light ray 301 enters the optical cavity, ittravels on a path that is folded inside the cavity, such that the pathincludes a first pass inside the cavity, then a first roundtrip in thecavity, and then a second roundtrip in the cavity, and so on; (b) thefirst pass comprises path 360 from the front STM 133 to point 340 in therear STM 135; (c) the first roundtrip comprises path 361 from point 340in the rear STM 135 to the front STM 133 and then path 362 from thefront STM 133 to point 341 in the rear STM 135; and (d) the secondroundtrip comprises path 363 from point 341 in the rear STM 135 to thefront STM 133 and then path 364 from the front STM 133 to point 342 inthe rear STM 135. The examples in the preceding sentence arenon-limiting examples. For instance, they are not the only first pass,first roundtrip and second roundtrip that occur in FIG. 3, and FIG. 3itself is a non-limiting example.

In FIG. 3, light rays in the optical cavity that are traveling towardthe rear STM 135 are aligned such that (if they were to pass through therear STM 135) they would focus at different rear focal planes, dependingon which round trip immediately preceded their exit from the cavity. InFIG. 3, the more roundtrips that occur before a portion of light exitsthe cavity, the closer the rear focal plane for that portion of light isto lens 131. Thus, one may control the lens/ToF camera distance byplacing a ToF camera at a rear focal plane that corresponds to a givenroundtrip of light (or to the first pass of light) in the opticalcavity.

In FIG. 3, the portion of light that exits through rear STM 135 after afirst pass inside the optical cavity is oriented such that it is focusedat rear focal plane 350. For example, light that travels in a first passalong path 360 and exits the cavity at the end of the first pass isfocused on rear focal plane 350, and intersects that plane at point 310.Rear focal plane 350 is located at distance fr0 from lens 131, which isequal to the nominal focal distance f0 of lens 131.

Likewise, in FIG. 3, the portion of light that exits through the rearSTM 135 after a first roundtrip inside the optical cavity is orientedsuch that it is focused at rear focal plane 351. For example, light thattravels in a first roundtrip along path 362 and exits the cavity at theend of the first roundtrip is focused on rear focal plane 351, andintersects that plane at point 311. Rear focal plane 351 is located atdistance fr1 from lens 131, which is less than fr0 and f0; that is,fr1<fr0=f0.

Likewise, in FIG. 3, the portion of light that exits through the rearSTM 135 after a second roundtrip inside the optical cavity is orientedsuch that it is focused at rear focal plane 352. For example, light thattravels in a first roundtrip along path 364 and exits the cavity at theend of the second roundtrip is focused on rear focal plane 352, andintersects that plane at point 312. Rear focal plane 352 is located atdistance fr2 from lens 131, which is less than fr1; that is,fr2<fr1<fr0=f0.

In the example shown in FIG. 3, if a ToF camera is located at plane 352,then light will arrive at the camera at different times, depending onwhether it exited the optical cavity after the first pass, the firstroundtrip or the second roundtrip. Specifically, in FIG. 3, if a ToFcamera is located at plane 352, then: (a) light that exits the opticalcavity after the first pass will reach the camera earlier than willlight that exits the cavity after the first roundtrip; and (b) lightthat exits the optical cavity after the first roundtrip will reach thecamera earlier than will light that exits the cavity after the secondroundtrip.

In the example shown in FIG. 3, if a ToF camera is located at plane 352at distance fr2 from the lens 131, then the light incident on the ToFcamera: (a) will appear focused if it exited the optical cavity at theend of the second roundtrip, and (b) will appear unfocused if it exitedthe optical cavity at the end of the first pass or first roundtrip inthe cavity. In FIG. 3, the ToF camera has a temporal resolution that isless than t3 minus t2, and thus: (a) the ToF camera can distinguishlight that exited the cavity at the end of the second roundtrip fromlight that exited at the end of other roundtrips or at the end of thefirst pass; and (c) thus, the ToF camera can (by measuring light thatexited the cavity at the end of the second roundtrip) image the scenewithout a loss of spatial resolution. For purposes of this paragraph, ifa pulse of light from the scene enters the optical cavity and reflectsrepeatedly inside the cavity, traveling in a first pass, firstroundtrip, second roundtrip, and so on in the cavity, then: (a) t3 isthe time at which a third exit pulse reaches plane 352; and (b) t2 isthe time at which a second exit pulse reaches plane 352 In the exampleshown in FIG. 3, the duration of a roundtrip in the cavity is equal tot3−t2.

In FIG. 3, the first, second and third exit pulses comprise light pulsesthat exit the optical cavity at the end of the first pass, firstroundtrip and second roundtrip, respectively, of light in the cavity.

In FIG. 3, if a ToF camera is placed at distance fr2 from the lens andhas a temporal resolution (e.g., shutter speed)) that is faster than theduration of a roundtrip of light in the optical cavity (or has a depthresolution that is finer than the roundtrip length of the cavity), thenthe camera may resolve the scene with no loss of spatial information.

In FIG. 3, if a ToF streak camera that captures x-t images is located atplane 352 and the scene being imaged comprises a point source of light,then the streak camera may capture an x-t image that shows an unfocusedfirst streak of light that corresponds to a first exit pulse from theoptical cavity, an unfocused second streak of light that corresponds toa second exit pulse from the cavity, and a focused point of light thatcorresponds to the third exit pulse from the cavity. The first streak,second streak and focused point of light will appear at differentpositions along the t (time) axis of the x-t image because theycorrespond to light that arrived at the streak camera at differenttimes. An example of this is shown in FIG. 6A.

In some cases, in an optical configuration such as that shown in FIG. 3:(a) the optical distances that light travels from the scene to points310, 311 and 312, respectively, are identical (due to folding of opticalpath inside the optical cavity); and (b) thus a pulse of light from thescene that enters the optical cavity will reach points 310, 311, 312 atthe same instant of time. Note: FIG. 3 is not drawn to scale.

In some implementations of this invention, compression of thelens/sensor distance (and thus a smaller form factor for the imagingsystem) is achieved as follows: (a) a “lens before optical cavity”configuration is employed, in which light from a scene passes through aconverging lens, then through an optical cavity, and then travels to aToF camera; (b) the ToF camera sensor is located at a geometric plane atwhich light (that passes through the lens, is folded in the opticalcavity, and exits the cavity at the end of a given roundtrip of light inthe cavity) is focused; (c) the physical Euclidean distance between thelens and the geometric plane is less than the nominal focal length ofthe lens; (d) the ToF camera has a temporal resolution that is less thanthe duration of a roundtrip of light in the cavity; and (e) the ToFcamera captures an image of the scene without loss of spatial resolution(as compared to the spatial resolution that would exist if the ToFcamera were located at a distance from the lens equal to the nominalfocal length of the lens.)

In a prototype of this invention, compression of the lens/sensordistance was achieved by a “lens before optical cavity” configuration.In this prototype: (a) an achromatic doublet 2″ lens with a focal lengthof 15 cm is placed in front of two 2″ O.D. 1.3 reflective ND filters;(b) the ND filters comprise an optical cavity; (c) light that passesthrough the lens is converging as it enters the cavity; (d) during eachround trip in the cavity, some of the light will exit and will bemeasured by a ToF camera; and (e) the light travels a different distancewith each round trip and is focused on a different plane.

In another prototype of this invention, compression of the lens/sensordistance was also achieved by a “lens before optical cavity”configuration. In this other prototype, a long focal length (25 cm)convex lens was placed in front of a streak camera at a distance of 10cm. This short distance between the streak camera and lens would cause aconventional camera to capture an unfocused image. However, in thisprototype, the streak camera captured a sharp focused image of a pointsource of light.

The prototypes described in the preceding two paragraphs arenon-limiting examples of this invention. This invention may beimplemented in many other ways.

In many implementations, it is desirable to compress the lens/sensordistance by employing a “lens before optical cavity” configuration. Forexample, compression of the lens-sensor distance may be strongly desiredfor applications where space and weight translates to cost or limitationof mobility. Spatial compression of the lens tube may also promotepractical use of large diameter long focal length lenses, because (insome cases in which a “lens before optical cavity” configuration isemployed), the compression may be achieved without any telephoto lensgroup or non-coaxial catadioptric lens systems.

FIG. 4 shows an example of hardware of a ToF imaging system, in anillustrative implementation of this invention. In FIG. 4: (a) a “lensbefore optical cavity” configuration is employed; and (b) the opticalcavity folds optical paths of light, thereby allowing compression of thelens/sensor distance (and thus a smaller form factor for the imagingsystem).

In FIG. 4, a pulsed laser 401 (Ti-Sapphire 780 nm 80 MHz rep. rate and30 fs pulse width) illuminates a scene. Pulses of the laser aresynchronized with image acquisition by a streak camera 425 (Hamamatsu®C5680). An achromatic doublet lens 431 with 15 cm focal length ispositioned in front of an optical cavity. The optical cavity comprisestwo semi-reflective, O.D. 1.3, neutral density (ND) filters 433, 435that are facing each other.

In the example shown in FIG. 4, a pulse of laser light from the laser401 is steered by mirror 403, then steered by mirror 405, then expandedby beam expander 407, then diffused by diffuser 409, and then passesthrough mask 411. This mask 411 comprises the scene that is imaged bystreak camera 425. After the pulse of light passes through the mask 411,the pulse is steered by periscope mirrors 415 and 412, then passesthrough lens 431, then passes through an optical cavity that comprisesND filters 433 and 435, and then travels to ToF camera 425. A portion ofthe light reflects repeatedly in the optical cavity, before exiting thecavity and traveling to the streak camera 425. A timer 483 outputs asignal that triggers the laser 401 to emit a laser pulse to illuminatethe scene and triggers the streak camera 425 to acquire a streak imageof the scene, such that the laser pulse and image acquisition aresynchronized. Memory devices 460, 461 and 462 are housed in the computer450, laser 401 and streak camera 425, respectively. In some cases, dataand instructions are transmitted by wired connections among the computer450, laser 401 and streak camera 425. In some cases, the computer 450,laser 401 and streak camera 425 include wireless communication modules470, 471, 472, respectively, and employ these modules for wirelesscommunication withe each other or with other devices. Computer 450controls and receives data from microcontrollers 480, 481 located instreak camera 425 and laser 401, respectively.

In the example shown in FIG. 4: (a) the streak camera captures an x-timage that measures spatial information in only one dimension (x); and(b) the set of periscope mirrors scans they dimension to obtain an x-y-tdata cube. Alternatively, one may avoid the y-scan by using a 2Dsingle-photo-avalanche diode array (SPAD) camera or by using lensletarrays to have 2D image acquisition.

In the examples shown in FIGS. 5A, 5B and 5C: (a) the entrance opticscomprise a lens 531, a front ND filter 533, and a rear ND filter 535;(b) the two ND filters together comprise a FP linear optical cavity; and(c) light 530 from the scene enters the entrance optics, passes throughthe entrance optics, and then travels to a ToF camera 125. Light 537 isthe light as it travels from the entrance optics toward the ToF camera.

FIG. 5A shows an example of a “lens before optical cavity” configurationof optical elements, in an illustrative implementation of thisinvention. In FIG. 5A, lens 531 is located in front of an opticalcavity, which comprises two ND filters 533, 535. The “lens beforeoptical cavity” configuration shown in FIG. 5A may be employed toshorten the physical distance between the lens and ToF camera sensor,and thereby achieve a smaller form factor for the ToF imaging system.

FIG. 5B shows an example of a “lens inside optical cavity” configurationof optical elements, in an illustrative implementation of thisinvention. In FIG. 5B: (a) lens 531 is located inside an optical cavity;and (b) the optical cavity comprises two ND filters 533, 535. The “lensinside optical cavity” configuration shown in FIG. 5A may be employedfor ultrafast multi-zoom imaging.

In a prototype of this invention, a “lens inside optical cavity”configuration is employed for multi-zoom imaging. In this prototype: (a)an achromatic doublet 2″ lens with a focal length of 15 cm is placed inbetween two 2″ O.D. 1.3 reflective ND filters; (b) the ND filterscomprise an optical cavity; (c) light enters the cavity and passesrepeatedly through the lens; (d) during (or at the end of) each roundtrip of light in the cavity, a portion of the light exits the cavity andis measured by a ToF camera. Within the optical cavity, a portion of thelight passes through the lens repeatedly, amplifying the focusing effectof the lens. The prototype described in this paragraph is a non-limitingexample of this invention; this invention may be implemented in manyother ways.

FIG. 5C shows an example of a “lens behind optical cavity” configurationof optical elements, in an illustrative implementation of thisinvention. In FIG. 5C: (a) lens 531 is located behind an optical cavity;and (b) the optical cavity comprises two ND filters 533, 535. The “lensbehind optical cavity” configuration shown in FIG. 5C may be employedfor ultrafast multi-spectral imaging.

In a prototype of this invention, a “lens behind optical cavity”configuration is employed for multi-spectral imaging. In this prototype:(a) a convex 2″ lens with a focal length of 7.5 cm is placed behind anoptical cavity that comprises a 2″ O.D. 1.3 reflective ND filter and a2″ O.D. 1.0 reflective filter; (b) light enters the optical cavity andexits through the lens that is focused on the camera; and (c) each roundtrip in the cavity adds a delay to the signal that exits. The “lensbehind optical cavity” configuration shown in FIG. 5C may be employedfor ultrafast multi-spectral imaging.

FIGS. 6A, 6B, 6C, and 6D are images captured by a streak camera (or arecreated by post-processing of images captured by a streak camera), in anillustrative implementation of this invention. The imaging system thatcaptured the streak images shown in FIGS. 6B, 6C, and 6D employed a“lens before optical cavity” configuration, in order to reducelens/sensor distance and thereby to achieve a smaller form factor. Forthe x-t streak images shown in FIGS. 6A, 6B, and 6C, the vertical axisis t (time) and time increases from the bottom to the top of the x-timage.

As used herein: (a) “first pass light” means light that exits an opticalcavity at the end of a first roundtrip of light in the cavity; (b)“first roundtrip light” means light that exits an optical cavity at theend of a first roundtrip of light in the cavity; (c) “second roundtriplight” means light that exits an optical cavity at the end of a secondroundtrip of light in the cavity; (d) “third roundtrip light” meanslight that exits an optical cavity at the end of a third roundtrip oflight in the cavity; (e) “fourth roundtrip light” means light that exitsan optical cavity at the end of a fourth roundtrip of light in thecavity; and (f) “fifth roundtrip light” means light that exits anoptical cavity at the end of a fifth roundtrip of light in the cavity;

FIG. 6A comprises an x-t streak image 601 of a scene that comprises apoint source of light. The streak camera that captured the streak imageshown in FIG. 6A was positioned at the geometric plane at whichsecond-roundtrip light is focused. In FIG. 6A: (a) the lowest prominentstreak of the streak image is an image (of the point source) that isunfocused in the x dimension and that is created by first pass light;(b) the second lowest, prominent streak in the streak image is an image(of the point source) that is unfocused in the x dimension and that iscreated by first roundtrip light; and (c) the point of light in themiddle of the streak image is a focused image (of the point source) thatis formed by second roundtrip light.

The streak camera that captured the streak images shown in FIGS. 6B, 6Cand 6D was positioned at the geometric plane at which fifth-roundtriplight is focused.

FIG. 6B comprises an x-t streak image 602 of an illuminated mask. Thefive long, prominent streaks of light in the streak image are images (ofthe mask) that are unfocused in the x dimension and that were created byfirst pass light, first roundtrip light, second roundtrip light, thirdroundtrip light and fourth roundtrip light, respectively. The dashedline at the top of the streak image is a focused image of the mask thatis formed by fifth roundtrip light.

FIG. 6C comprises five different, unfocused, x-y images 603 of a mask.The five images are so unfocused that the details of the mask are notdiscernible. In FIG. 6C, the five unfocused images are formed by firstpass light, first roundtrip light, second roundtrip light, thirdroundtrip light and fourth roundtrip light.

FIG. 6D shows a focused x-y image 604 of the same mask (which spells theletters “MIT”). In FIG. 6D, the focused image is formed by fifthroundtrip light

FIG. 7 shows a top view of a multi-zoom ToF imaging system, in anillustrative implementation of this invention. In FIG. 7, a “lens insideoptical cavity” configuration enables ultrafast, multi-zoom imaging.

In the example shown in FIG. 7, the optical cavity comprises STM 733 andSTM 735. This optical cavity is located in front of a ToF camera 725.Lens 731 is located inside the optical cavity. After light from thescene enters the cavity, light passes through lens 731 (which is insidethe cavity) repeatedly, such as during the first pass of light insidethe cavity and during each roundtrip of light inside the cavity,respectively. As light reflecting inside the cavity passes through lens731 repeatedly, the light is further diffracted during each pass oflight through lens 731. Thus, each pass of light through lens 731 causesan optical system (comprising the lens 731 and optical cavity) to befocused at a different depth of the scene, because each pass of lightthrough the lens changes the convergence or divergence of the wavefront.

In FIG. 7, lens 731 (which is inside the optical cavity formed by STM733 and STM 735) enables the ToF camera 725 to function as multi-zoomultrafast camera that is configured to focus on two or more focal planesat different depths (e.g., depths that differ by an order of magnitudeor more) with different magnification factors. For example, in FIG. 7,the ToF camera 725 employs the same lens 731 to focus on object 751 andto focus on object 761 (even though they are at different scene depths),without physically moving any optical element and without changing theshape of any lens. In FIG. 7, the ToF camera may capture a multi-zoomimage in a single ultrafast acquisition frame using a single set ofoptical elements, without physically altering the focal length of thelens and without physically moving any optical element.

In FIG. 7, the focal length of lens 731 is larger than the opticalcavity size, in order to increase the stability of the cavity in theinitial roundtrips.

In some implementations (in which a “lens inside optical cavity”configuration is used for multi-zoom imaging), increasing the number ofroundtrips in the optical cavity (before a portion of light exits thecavity): (a) tends to reduce the scene depth at which the ToF camera isfocused in an image formed by that portion of light; and (b) tends toincrease the magnification of that image.

The following paragraph is a description of a prototype of thisinvention.

In this prototype, the configuration shown in FIG. 7 was employed (keepin mind that FIG. 7 is not drawn to scale). In this prototype: (a) lens731 had a nominal focal length of 15 cm; (b) the optical cavity size(distance between front STM 733 and rear STM 735) was 4 cm; (c) object751 was located 4 cm in front of lens 731; and (d) object 761 waslocated 55 cm in front of lens 731. In this prototype, ToF camera 725comprised a streak camera. The streak camera captured two focusedimages—a first image of a more distant object 761 and a second image ofcloser object 751—using the same lens 731 and without moving anyphysical optical elements and without changing the shape of any lens.The first image (of distant object 761) was formed by first roundtriplight and the second image (of close object 751) was formed by secondroundtrip light. The magnification in the second image (of close object751) was approximately four times greater than the magnification in thefirst image (of distant object 761). In this prototype, there was alarge temporal separation (hundreds of picoseconds) between the time ofarrival at the streak camera of the light that created the first andsecond images, respectively, and thus there was no artifact or tracefrom the first image onto the second image.

The prototype described in the preceding paragraph is a non-limitingexample of this invention; this invention may be implemented in manyother ways.

In some implementations, multi-zoom acquisition may be advantageous inimaging (e.g., microscopy or remote sensing) where one wants to see theentire field to find the region of interest faster while holding ontothe details of the zoomed-in image during the same acquisition with nomechanical movements of lenses.

FIG. 18A shows a top view of a multi-spectral ToF imaging system, in anillustrative implementation of this invention. In FIG. 18A, a set ofoptical cavities enables ultrafast, multi-spectral imaging bytime-encoding different spectral signals.

In the example shown in FIG. 18A, an ND filter and multiple notchfilters create a set of optical cavities. The notch filters may bepositioned in a “stack”, one in front of the other, such that they havea uniaxial configuration (share a common axis).

In FIG. 18A, four filters 1801, 1833, 1835, 1837, a lens 1831 and a ToFcamera 1825 share a common optical axis 1890.

In the example shown in FIGS. 18A, 18B and 18C, multi-spectral light1800 from the scene enters a broadband ND filter 1801 that transmitslight in a broad band 1850 of light. Then the light transmits throughthree notch filters 1833, 1835, 1837. Each of these notch filters, 1833,1835, 1837 transmits substantially all incident light in frequencies inbroad band 1850, except for a notch (subband) of frequencies. There is adifferent subband for each notch filter. Specifically, filter 1833reflects a majority of incident light in subband 1853 and transmits aminority of incident light in subband 1853. Filter 1835 reflects amajority of incident light in subband 1855 and transmits a minority ofincident light in subband 1855. Filter 1837 reflects a majority ofincident light in subband 1857 and transmits a minority of incidentlight in subband 1857.

In FIG. 18A, the notch filters 1833, 1835 and 1837 are spectral filters.The amount of light that they each reflect or transmit depends on thefrequency of light incident on them.

FIG. 18B and FIG. 18C are charts that show an example of transmittanceand reflectance, respectively, for these three notch filters in thismulti-spectral system. In FIG. 18B, lines 1863, 1865, 1867 representlight that is transmitted by filters 1833, 1835 and 1837, respectively.These lines 1863, 1865, 1867 overlap at many wavelengths, as shown nearthe top of FIG. 18B. Likewise, in FIG. 18C, lines 1873, 1875, 1877represent light that is reflected by filters 1833, 1835 and 1837,respectively. These lines 1873, 1875, 1877 overlap at many wavelengths,as shown at the bottom of FIG. 18C.

In the example shown in FIGS. 18A, 18B and 18C, filters 1833, 1835 and1837 each transmit substantially all incident light in frequencies inbroad band 1850, except for a notch (subband) of frequencies. The notchis different for each of these three filters 1833, 1835, 1837. Forexample, filter 1833: (a) reflects 90% of incident light in subband1853; (b) transmits 10% of incident light in subband 1853, and (c)transmits substantially all incident light in the remainder of the broadband 1850 (including substantially all incident light in subbands 1855and 1857). Likewise, filter 1835: (a) reflects 90% of incident light insubband 1855; (b) transmits 10% of incident light in subband 1855, and(c) transmits substantially all incident light in the remainder of thebroad band 1850 (including substantially all incident light in subbands1853 and 1857). Likewise, filter 1837: (a) reflects 90% of incidentlight in subband 1857; (b) transmits 10% of incident light in subband1857, and (c) transmits substantially all incident light in theremainder of the broad band 1850 (including substantially all incidentlight in subbands 1853 and 1855).

For clarity of presentation, the examples in FIGS. 18B and 18C assumethat the internal absorbance of the filters is zero. However, in actualpractice, internal absorbance of the filters may be non-zero.

FIGS. 18B and 18C are non-limiting examples. This invention is notlimited to the percentages of transmitted light and reflected lightshown in FIGS. 18B and 18C.

In the example shown in FIG. 18A, the ND filter 1801 and the three notchfilters 1833, 1835, 1837 form three optical cavities. Specifically,filter 1801 and filter 1833 comprise a first optical cavity. Filter 1801and filter 1835 comprise a second optical cavity. Filter 1801 and filter1837 comprise a third optical cavity.

These three optical cavities have different sizes. Specifically, thefilter-to-filter distance for the first optical cavity is shorter thanthat of the second optical cavity, which is shorter than that of thethird optical cavity.

Thus, the roundtrip distance (and temporal duration of a roundtrip)inside each of these three cavities is different. Specifically, theroundtrip distance (and temporal duration of a roundtrip) for the firstoptical cavity is shorter than that of the second optical cavity, whichis shorter than that of the third optical cavity.

As a non-limiting example, if subbands 1853, 1855 and 1857 are green,blue, and red light, respectively, then: (a) a portion of the greenlight that enters the first optical cavity (comprising ND filter 1850and filter 1833) will reflect repeatedly in the first optical cavity;(b) a portion of the blue light that enters the second optical cavity(comprising ND filter 1850 and filter 1835) will reflect repeatedly inthe second optical cavity; and (c) a portion of the red light thatenters the third optical cavity (comprising ND filter 1850 and filter1837) will reflect repeatedly in the third optical cavity.

However, each time that light is incident on filters 1833, 1835, 1837,respectively, a portion of the light will be transmitted and travel tolens 1831 and ToF camera 1825. This is because filters 1833, 1835 and1835 transmit almost all light outside of their respective notches(subbands) and transmit a minority of light even in their respectivenotch (subband).

In illustrative implementations, the three optical cavities havedifferent sizes and thus different roundtrip distances. Because theoptical cavities have different sizes (roundtrip distances), theytime-encode spectral information.

For example, in FIG. 18A, if an impulse of multi-spectral light from thescene strikes the ND filter 1801, then different subbands of frequencieswill be time-encoded, in that they will arrive at the ToF camera atdifferent times.

As a non-limiting example, if subbands 1853, 1855 and 1857 are green,blue, and red light, respectively, then in response to an impulse ofmulti-spectral light from the scene striking the ND filter 1801: (a) aset of exit pulses of green light would exit the first optical cavity,then travel through filters 1835 and 1837, then travel through lens1831, and arrive at ToF camera 1825; (b) a set of exit pulses of bluelight would exit the second optical cavity, then travel through filter1837, then travel through lens 1831, and arrive at ToF camera 1825; (c)a set of exit pulses of red light would exit the third optical cavity,then travel through lens 1831, and arrive at ToF camera 1825; (d) theearliest of the green exit pulses would arrive at ToF camera 1825 beforethe earliest of the blue exit pulses; (e) the earliest of the blue exitpulses would arrive at ToF camera 1825 before the earliest of the redexit pulses; and (f) ToF camera 1825 would have a sufficiently fasttemporal resolution to distinguish between the earliest blue, green andred exit pulses, respectively. In the example described in the precedingsentence: (a) the different-sized optical cavities would cause spectralinformation to be time-encoded, by causing green, blue and red exitpulses, respectively, to arrive at ToF camera 1825 at different times;and (b) these different times would be resolvable by ToF camera 1825 dueto its fast temporal resolution.

In some implementations, a set of notch filters may be at differentdistances from an ND filter, and thus they may create different sizeoptical cavities and may encode each center wavelength to a differenttime. Thus, the ToF camera may measure each selected frequency at adifferent time offset. In the x-y-t datacube the different spectralimages may exist on different time planes. The number of notch filtersto use may depend on the number of spectral images desired.

The uniaxial design shown in FIG. 18A has many advantages. Among otherthings, the uniaxial design is easy to scale. For example, in somecases, more notch filters may be added (behind or in front of thoseshown in FIG. 18A) to measure for additional wavelengths without theneed to change any framing of the lens tube. Also, using three notchfilters at red, green, and blue, respectively, may allow a ToF camera tocapture ultrafast, single acquisition, color ToF images. Such colorimages may be desirable in many applications, including ultrafastfluorescence lifetime imaging. Furthermore, as compared to the“side-by-side” design shown in FIG. 8 (discussed below), the uniaxialdesign in FIG. 18A: (a) may tend to be more robust against chromaticaberrations; and (b) may enable more light to reach the lens for eachfrequency, because (in the uniaxial design), the aperture is not dividedfor side-by-side spectral filters.

FIG. 18A shows three notch filters and three optical cavities. However,the number of notch filters and number of optical cavities (e.g., in auniaxial configuration for multi-spectral imaging) may vary, dependingon the particular implementation. For example, in a uniaxialconfiguration for multi-spectral imaging: (a) the number of notchfilters may be equal to the number of optical cavities; and (b) thenumber of optical cavities may be two, three, four, five, six, seven,eight, nine, ten, or more than ten and less than twenty, or more than orequal to twenty.

In some cases, time-encoded spectrums from the different spectralfilters enable the ToF camera to perform single-acquisitionmulti-spectral imaging at ultrafast rates. For example, in some cases(including implementations shown in FIGS. 18 and 8, time-encodedspectrums may enable the ToF to perform multi-spectral imaging at thenominal temporal resolution of the ToF camera, with no loss of spatialresolution and with no loss of temporal resolution. This is much betterthan conventional methods of spectral filtering, which: (a) use eitherelectronic illumination sweeping or dividing the sensor's spatialresolution for different wavelengths; and (b) thus may limit speed (ofimage acquisition) and resolution of the acquired image.

Ultrafast multi-spectral imaging has many practical applications. Amongother things, it may be used for fluorescent imaging of biologicalsamples as well as imaging of non-reversible phenomena such as ablation,ionization, and filamentation.

Alternatively, in some implementation of this invention, amulti-spectral ToF imaging system may be positioned side-by-side (asshown in FIG. 8), instead of in a uniaxial configuration.

FIG. 8 shows a top view of a multi-spectral ToF imaging system, in anillustrative implementation of this invention. In FIG. 8, a set ofoptical cavities enables ultrafast, multi-spectral imaging bytime-encoding different spectral signals.

In FIG. 8, multi-spectral light 830 passes through a first spectralfilter 821 and a second spectral filter 822. These spectral filters 821,822 are located side-by-side. Thus, light from both spectral filtersreaches lens 831, but light that passes through the first spectralfilter does not pass the second spectral filter, and light that passesthrough the second spectral filter does not pass through the firstspectral filter. The first spectral filter 821 allows a first spectrumof light to pass (e.g. a narrow band of wavelengths). The secondspectral filter 822 allows a second spectrum of light to pass (e.g., adifferent, narrow band of wavelengths).

In FIG. 8, a first optical cavity is located in an optical path betweenthe first spectral filter 821 and the lens 831. Specifically, the firstoptical cavity is behind the first spectral filter 821 and in front oflens 831. The first optical cavity comprises STM 833 and STM 835.

In FIG. 8, a second optical cavity is located in an optical path betweenthe second spectral filter 822 and the lens 831. Specifically, thesecond optical cavity is behind the second spectral filter 822 and infront of lens 831. The second optical cavity comprises STM 843 and STM845.

In FIG. 8, the first optical cavity has a different roundtrip distance(distance that light travels during a roundtrip in the cavity) than thesecond optical cavity does. For example, if an impulse of light from thescene enters the first and second optical cavities, then: (a) exitpulses from the first optical cavity occur at different times than exitpulses from the second optical cavity; and (f) thus, the first andsecond spectrums of light (from the first and second spectral filters,respectively) arrive at the ToF camera at different times.

In FIG. 8, the first and second optical cavities, respectively, delaythe first and second spectrums by different amounts of time due to thedifferent roundtrip distances in the cavities. Thus, in FIG. 8, a set ofoptical cavities time-encodes different spectrums from differentspectral filters.

Alternatively, in FIG. 8, each of the spectral filters 821, 822 may belocated behind (instead of in front of) its respective optical cavity.For example, in some alternative implementations, filter 821 may belocated behind the optical cavity comprising STMs 833 and 835, andfilter 822 may be located behind (instead of in front of) the opticalcavity comprising STMs 843 and 845.

Alternatively, in FIG. 8, each of the spectral filters may be locatedinside (instead of in front of or behind) an optical cavity. Forexample, in some alternative implementations, filter 821 may be locatedinside the optical cavity comprising STMs 833 and 835, and filter 822may be located inside the optical cavity comprising STMs 843 and 845.Placing a spectral filter inside an optical cavity—such that lightreflecting repeatedly inside the cavity passes through spectral filterrepeatedly—may tend to sharpen the spectral window of the filter in anexponential manner, because the light signal passing through the filtermay be multiplied by the filter each time that light passes through thefilter. Sharpening the filter window may be desirable, for example, inspectrum un-mixing of biological samples stained with fluorophores withneighboring spectra.

In illustrative implementations of this invention, light exiting theoptical cavity becomes fainter as the number of roundtrips increases.That is, the intensity of light exiting the cavity tends to decrease asthe number of roundtrips (before the light exits the cavity) increases.

FIG. 9 is a chart of transmission intensity as a function of number ofoptical cavity round trips, in a prototype of this invention. In FIG. 9,the vertical axis is transmission intensity (that is, the intensity oflight exiting the optical cavity) and the horizontal axis is the numberof folds. For purposes of FIG. 9: (a) there are two folds per roundtripof light in the cavity and (b) the cavity is a Fabry-Perot opticalcavity that comprises two ND filters. The optical density shown in thekey in FIG. 9 is the optical density of each ND filter, respectively, inthis Fabry-Perot cavity.

FIG. 9 illustrates that, for an optical cavity with relatively lowoptical density ND filters (e.g., optical density equal to 0.75): (a)the initial drop-off in intensity is relatively small, but (b) theintensity then drops off very rapidly thereafter. FIG. 9 illustratesthat, for an optical cavity with relatively high optical density NDfilters (e.g., optical density equal to 2.0): (a) the initial drop-offin intensity is relatively large, but (b) the intensity then drops offvery slowly thereafter. For example, an ND filter with an opticaldensity of 2.0 transmits 1% of incident light and reflects 99% ofincident light. Thus, for a ND filter with an optical density of 2.0:(a) there is a large drop-off of intensity initially, because only 1% ofthe light is transmitted through the ND filter at the end of the firstpass of light inside the cavity, and (b) the drop-off of intensity isvery slow thereafter, because only 1% of the then remaining light leaksfrom the cavity in each reflection from an ND filter thereafter.

In many implementations, the drop-off in intensity as roundtripsincrease creates a design tradeoff. For example, if the goal is todecrease lens/sensor distance by folding light inside an optical cavityusing a “lens before optical cavity” configuration, then increasing thenumber of roundtrips: (a) tends to decrease the lens/sensor distance(which is desirable) but (b) tends to reduce the intensity of light(which is often not desirable).

The drop-off of intensity (as number of roundtrips increase) tends notto be a problem in some implementations, where ultrafast ToF camerasthat operate in a photon-starved mode (e.g., SPAD cameras) are employed.

Furthermore, the drop-off of intensity (as the number of roundtripsincrease) tends to be mitigated in some implementations, in which imagesmay be acquired with only one or a few roundtrips. For example, in someimplementations, a multi-spectral ToF may capture time-encoded spectraldata after only one or two roundtrips. Also, in some cases, a multi-zoomcamera that focuses at x number of scene depths may capture images withonly x−1 roundtrips (e.g., by capturing an image of first pass light, animage of first roundtrip light, and so on, until x images, each focusedat a different scene depth, are captured). For example, in some cases, amulti-zoom camera that focuses at two scene depths may capture imageswith only one roundtrip in the cavity.

FIG. 10 shows a ToF imaging system that takes ellipsometry measurements,in an illustrative implementation of this invention. In FIG. 10, a ringoptical cavity enables ultrafast ellipsometry measurements.

In the example shown in FIG. 10, a ring optical cavity is located in anoptical path between a scene and a ToF camera. Light from a scene 1001passes through a periscope mirror set 1012 and strikes STM 1033. Then aportion of this light reflects from STM 1033 (without ever entering thering optical cavity) and travels to a linear polarizer (LP) 1041.Another portion of this light enters the ring optical cavity by passingthrough STM 1033, and then starts a roundtrip inside the cavity. Duringeach roundtrip of light in the ring optical cavity, light travels fromSTM 1033, to mirror 1021, then through half-wave plate (HWP) 1023, thento mirror 1024, then to mirror 1027 and then back to STM 1033. At theend of each roundtrip: (a) a portion of the light that strikes STM 1033passes through STM 1033 and travels to the linear polarizer 1041, and(b) a portion of the light that strikes STM 1033 reflects from STM 1033and starts another roundtrip in the cavity. Thus, after light enters thering optical cavity, it reflects in repeated roundtrips in the cavity,with part of the light leaking through STM 1033 at the end of eachroundtrip.

In FIG. 10, after light reflects from STM 1033 (without entering thecavity) or exits the cavity (by passing through STM 1033), the lighttravels to linear polarizer 1041, then to lens 1031, then to ToF camera1025.

In FIG. 10, the half-wave plate (HWP) 1023 rotates the polarization oflight, each time that light passes through the half-wave plate (e.g.,during each roundtrip of light inside the optical cavity). Because thepolarization of light is rotated by a small angle in each roundtrip, thepolarization of light exiting the STM (and striking the linearpolarizer) is different for different roundtrips. Thus, after eachroundtrip in the cavity, the fixed linear polarizer filters a differentportion of the polarization spectrum of the initial light that enteredthe cavity (where each portion of the polarization spectrum comprises adifferent polarization angle or different range of polarization angles).In some implementations, this configuration (half-wave plate inside ringcavity) enables the ToF camera to take ellipsometry measurements.

In the example shown in FIG. 10, each time that the light passes throughHWP 1023, the HWP 1023 rotates the polarization angles of the light byangle a. In FIG. 10, linear polarizer (LP) 1041 allows only light withpolarization angle to pass through the LP. But the polarization of lightis being rotated by angle a each time that the light passes through theHWP.

Thus, in FIG. 10, in the case of light that never enters the ring cavity(but instead reflects off STM 1033 directly to the LP, and thus nevergoes through the HWP and is never rotated by the HWP), the LP allows topass (through the LP) only light that initially (i.e., when leaving thescene) had a polarization angle of β. In FIG. 10, in the case of lightthat completes only one roundtrip in the ring cavity (and thus travelsthrough the HWP once and is rotated by angle α), the LP allows to pass(through the LP) only light that initially (i.e., when leaving thescene) had a polarization angle of β−α. In FIG. 10, in the case of lightthat completes only two roundtrips in the ring cavity (and thus travelsthrough the HWP twice and is rotated by a cumulative angle of 2α), theLP allows to pass (through the LP) only light that initially (i.e., whenleaving the scene) had a polarization angle of β−2α.

Thus, in FIG. 10, the ToF camera 1025 may capture multiple images, whereeach image is of light with a different initial polarization angle (ordifferent range of polarization angles). For example, the ToF camera maycapture images of light that has completed zero, one or more roundtripsof light in the ring cavity, and thus has different initial polarizationangles (or ranges of polarization angles), depending on the number ofroundtrips completed.

By capturing data regarding light with different polarization angles,the ToF camera 1025 may take ellipsometric measurements in a singleacquisition. Advantageously, in some cases, these ellipsometricmeasurements may be ultrafast and wide-field (and thus not taken by apoint-by-point raster scan). In some implementations of this invention,the polarization evolution of ultrafast phenomena may be captured withthe nominal time-resolution of the ToF camera 1025.

In FIG. 10, if the ToF camera 1025 is a streak camera that measuresspatial information in only an x dimension, then the periscope mirrors1012 may be employed to scan the scene in they dimension. In othercases, the periscope mirrors 1012 may be omitted, and light from thescene 1001 may travel directly to STM 1033.

FIG. 11 is a diagram that shows an example of a Fourier region and afractional Fourier region of an optical system.

The optical configuration in FIG. 11 is commonly referred to as a “4 f”configuration. In FIG. 11, a transmissive scene (e.g., a backlit mask orbacklit transparency) is located at plane 1101. The transmissive sceneincludes points 1120 and 1122. A spherical lens 1131 is located at plane1102. Another spherical lens 1133 is located at plane 1104. Both ofthese lenses have a focal length f The total distance from plane 1101 toplane 1105 is 4 f that is, plane 1101 is distance f from plane 1102,which is distance f from plane 1103, which is distance f from plane1104, which is distance f from plane 1105.

In the example shown in FIG. 11, a transmissive scene is backlit bycollimated light 1190 (i.e., by a plane wavefront of light). Thetransmissive scene includes points 1120, 1122 and is located at plane1101. Light that exits the transmissive scene travels through lens 1131,then through lens 1133, and then to plane 1105. An output image forms atplane 1105. As the light travels from plane 1101 to plane 1105, thefirst lens 1131 performs a Fourier transform (in the spatial frequencydomain) and the second lens 1133 performs an inverse Fourier transformthat produces, at the output plane 1105, an inverted replica of theinput image.

In the example shown in FIG. 11, let f(x,y) denote the intensity oflight in the 2D input scene at plane 1101. Then, under certainconditions: (a) the light signal at the Fourier plane (which is plane1103) is the Fourier transform (in the spatial frequency domain) off(x,y); and (b) the image that forms at the output plane (plane 1105) is−f(x,y); that is, the inverse of the input image. For example, in thisinverse image in the output plane, points 1132 and 1130 may correspondto points 1122 and 1120, respectively, of the input image. Each point inthe Fourier plane 1103 may correspond to a single spatial frequency.

In FIG. 11, the input light signal occurs at plane 1101.

In FIG. 11, the light signal in the region between planes 1102 and 1103may be modeled as a fractional Fourier transform of the input lightsignal. Likewise, the light signal in the region between planes 1103 and1104 may be modeled as a fractional Fourier transform of the input lightsignal. Thus, these two regions (between planes 1102 and 1103, andbetween planes 1103 and 1104, respectively) are examples of fractionalFourier regions.

FIG. 11 is included solely to give the reader an introduction to theconcepts of a Fourier plane and a fractional Fourier region. In mostimplementations of this invention: (a) a “4 f” configuration is notused; and (b) thus, the configuration shown in FIG. 11 is not used.

However, in many implementations of this invention, a Fourier planeexists one focal length from a spherical lens and a fractional Fourierregion exists, albeit in a different optical configuration than thatshown in FIG. 11.

For example, in some implementations of this invention: (a) a “lensbefore optical cavity” is employed; and (b) the optical cavity islocated in a fractional Fourier region.

FIG. 12 shows a ToF imaging system that performs ultrafast filtering ofspatial frequencies, in an illustrative implementation of thisinvention. In FIG. 12, a collimating lens outputs collimated light that,while collimated (i) passes through the Fourier plane and (ii) isincident on an optical cavity. This incident light then enters theoptical cavity.

In the example shown in FIG. 12, light from a scene 1201 may passthrough entrance lens 1240, then through collimating lens 1241, thenthrough an optical cavity, then through imaging lens 1231, and then to aToF camera 1225. The optical cavity may be a Fabry-Perot cavity thatcomprises STM 1233 and STM 1235. A mask 1250 may be located inside theoptical cavity.

In FIG. 12, entrance lens 1240 may converge the incoming wavefront.Plane 1280 may be a Fourier plane of the wavefront that exits lens 1240.This Fourier plane 1280 may be located at a distance from collimatinglens 1241 which is equal to the focal length of collimating lens 1241.Thus, light exiting lens 1241 may be collimated and may, while stillcollimated, pass through a Fourier plane and then be incident of theoptical cavity (which itself comprises STMs 1233 and 1235). For example,the collimated light may, while still collimated, be incident on thefront STM 1233 of the optical cavity. This incident light may then enterthe optical cavity. One of the surfaces of the optical cavity (e.g., STM1235) may be tilted relative to what would, in the absence of theoptical cavity, be an optical axis of the system (e.g., axis 1270). Thistilt may cause the optical cavity to be unstable. (In FIG. 12, the tiltis exaggerated for clarity of illustration).

As light reflects repeatedly inside the unstable cavity: (a) light maytend to “walk off” (e.g., tend to move further away from axis 1270); and(b) thus different regions of the light signal may be filtered (blocked)by mask 1250 in different roundtrips of light inside the cavity. In FIG.12, because each point in a Fourier plane corresponds to a differentspatial frequency, different spatial frequencies of light may befiltered (blocked) by mask 1250 in different roundtrips of light in theoptical cavity. At the end of the first pass of light in the cavity andat the end of each roundtrip of light in the cavity, respectively, lightinside the cavity may strike rear STM 1235. A portion of this light mayreflect from STM 1235 back into the FP cavity; and a portion may passthrough rear STM 1235. Light that exits the FP cavity through rear STM1235 may then travel through imaging lens 1231 to ToF camera 1225. Lens1231 may convert the filtered Fourier signal that exits the opticalcavity into a 2D spatial signal at the image plane (which is the sensorplane of the ToF camera).

Thus, in FIG. 12, different spatial frequencies may be filtered,depending on how many roundtrips in the optical cavity occur beforelight exits the cavity.

In some implementations: (a) an unstable optical cavity is located in a“Fourier volume” (e.g., 1260); and (b) a ToF camera performs ultrafastfiltering of spatial frequencies in a light signal. This ultrafastfiltering has many practical applications, including enhanced objectdetection in imaging of ultrafast phenomena, such as ultrafast fluidstreams or ultrafast fluid turbulences.

FIG. 13 is a flowchart of an image processing method, in an illustrativeimplementation of this invention. The method shown in FIG. 13 includesthe following steps: An x-y-t data cube is captured by a ToF camera(Step 1301). Map time axis to cavity round trips by observing peakintensities in time (Step 1302). Render x-y image for correspondinground trip times (Step 1303).

FIG. 14 is a flowchart of a method in which light passes through a “lensbefore optical cavity” configuration of optical elements, in anillustrative implementation of this invention. The method shown in FIG.14 includes the following steps: A laser pulse is generated andilluminates the scene (Step 1401). The light goes through a lens into acavity (Step 1402). In each roundtrip, some light remains in the cavityand some is transmitted (Step 1403). Transmitted light is measured by aToF camera (Step 1404).

FIG. 15 is a flowchart of a method in which light passes through a “lensinside optical cavity” configuration of optical elements, in anillustrative implementation of this invention. The method shown in FIG.15 includes the following steps: A laser pulse is generated andilluminates the scene (Step 1501). The light enters a cavity and passesthrough a lens in the cavity (Step 1502). In each roundtrip, some lightremains in the cavity (passing through the lens multiple times) and someis transmitted (Step 1503). Transmitted light is measured by a ToFcamera (Step 1504).

FIG. 16 is a flowchart of a method in which light passes through a “lensafter optical cavity” configuration of optical elements, in anillustrative implementation of this invention. The method shown in FIG.16 includes the following steps: A laser pulse is generated andilluminates the scene (Step 1601). The light enters the cavity. In eachroundtrip, some light remains in the cavity and some is transmitted(Step 1602). The transmitted light passes through a lens and is measuredby a ToF camera (Step 1603).

FIG. 17 shows an example in which spatial divergence of light increasesas the number of cavity roundtrips increases, in an illustrativeimplementation of this invention. In some implementations, thisincreasing spatial divergence makes it easier to detect small angulardifferences in the orientation of light incident on a ToF camera.

FIG. 17 is an x-t streak image 1701, in which streaks are formed bynon-paraxial light from an object in the scene. In FIG. 17: (a) thevertical axis in FIG. 17 is the t (time) dimension, with time increasingfrom top to bottom of the streak image; (b) line 1702 corresponds to anoptical axis of the imaging system; (c) non-paraxial light entered anoptical cavity and reflected repeatedly in the cavity; (d) as thisnon-paraxial light exited the cavity (at the end of the first pass inthe cavity and thereafter at the end of successive roundtrips in thecavity), this non-paraxial light formed a series of streaks in thestreak image; and (e) line 1703 intersects this series of streaks. InFIG. 17, the spatial distance between line 1703 (which intersects thestreaks formed by the non-paraxial light) and line 1702 (whichcorresponds to the optical axis) increases over time (the more timepasses, the greater the spatial divergence).

In some implementations, this increasing spatial divergence makes iteasier to detect small angular deviations of light from an optical axis.For example, in some use cases: (a) small angular deviations (relativeto an optical axis) of incident light at a ToF camera may be caused bytiny irregularities in a surface; and (b) the increasing spatialdivergence in the captured images may make it easier to detect thesesurface irregularities.

As noted above, in many implementations: (a) a light source (e.g., 101,401) emits pulsed light that illuminates the scene; (b) pulses of lightfrom a scene enter the optical cavity; (c) light reflects repeatedlyinside the optical cavity, with a portion of the light exiting at theend of the first pass of light in the cavity and thereafter at the endof successive roundtrips of light; and (e) the light exiting the opticalcavity forms a series of pulses of light (“exit pulses”).

However, this invention is not limited to pulsed light. In someimplementations, the light source (e.g. 101, 401) emits light that isnot pulsed, and light (from the scene) which is not pulsed enters theoptical cavity.

In some implementations of this invention, the ToF camera (e.g., 125,425, 725, 825, 1025, 1225, or 1825) comprises a streak camera thatcaptures data regarding time-of-arrival (or time-of-flight) of incidentlight. For example, the time-of-arrival (or time-of-flight) of incidentlight may be indicated by the t (time) dimension of an x-t imagecaptured by a streak camera.

However, this invention is not limited to a streak camera.

In illustrative implementations of this invention, any type of ToFcamera may be employed to measure: (a) time-of-flight of incident light;(b) time-of-arrival of incident light; (c) depth; or (d)scene-roundtrip-distance. As used herein, “scene-roundtrip-distance”means a distance that light travels from the camera to the scene andback to the camera. Scene-roundtrip-distance may be twice the depth.These factors (time-of-flight, time-of-arrival, depth, andscene-roundtrip-distance) may encode closely related information,because the distance that light travels along a given path depends (atleast in part) on how long it takes for the light to travel the path.

How many roundtrips of light occur in an optical cavity, before light(from the scene) exits the cavity and travels to the ToF camera, affectsall four of these factors (time-of-flight, time-of-arrival, depth, andscene-roundtrip-distance).

In illustrative implementations, any one or more of these four factors(time-of-flight, time-of-arrival, depth, and scene-roundtrip-distance)may be used to selectively acquire data regarding light (from the scene)that exited the cavity after a certain number of roundtrips in thecavity.

For example, in some implementations: (a) a “lens before cavity”configuration is employed; (b) light from the scene enters an opticalcavity; (c) the cavity folds an optical path, thereby allowing reductionof the Euclidean physical distance between a lens and ToF camera sensor;(c) the ToF camera sensor is located at a geometric plane at whichlight—which has exited the cavity after a given number of roundtrips inthe cavity—is focused; and (d) the ToF camera may employ any one or moreof these four factors (time-of-flight, time-of-arrival, depth, andscene-roundtrip-distance) to selectively acquire data regarding lightthat exited the cavity after the given number of roundtrips in thecavity.

Likewise, in some implementations: (a) a “lens inside cavity”configuration is employed; (b) light from the scene enters an opticalcavity; (c) a lens is located inside the cavity, such that light passesthrough the lens repeatedly as light reflects in roundtrips inside thecavity; (d) the divergence or convergence of the light wavefront insidethe cavity changes each time that light passes through the lens, andthus the portion of the wavefront that exits the cavity at the end ofeach roundtrip in the cavity, respectively, is focused at a differentscene depth (e.g., such that the scene depth at which the system isfocused changes after each roundtrip of light in the cavity); (e) theToF camera may employ any one or more of these four factors(time-of-flight, time-of-arrival, depth, and scene-roundtrip-distance)to selectively acquire a first set of data regarding light that exitedthe cavity after a first number of roundtrips in the cavity and toselectively acquire a second set of data regarding light that exited thecavity after a second number of roundtrips in the cavity; (f) the ToFcamera generates, based on the first set of data, a focused image at afirst scene depth; and (g) the ToF camera generates, based on the secondset of data, a focused image at a second scene depth. For example, thesecond scene depth may be more than twice the first scene depth.

Likewise, in some implementations: (a) a “lens behind cavity”configuration is employed for ultrafast multispectral imaging; (b) theimaging system includes multiple optical cavities; (c) light from thescene passes through the cavities; (d) each respective cavity is adifferent size and has a different roundtrip distance for lightreflecting in a complete roundtrip inside the respective cavity; (e)different colors (frequency bands) of light exit the set of opticalcavities at different times; and (f) the ToF camera may employ any oneor more of these four factors (time-of-flight, time-of-arrival, depth,and scene-roundtrip-distance) to selectively acquire a first set of dataregarding a first color of light that exits the set of optical cavitiesat a first time and to selectively acquire a second set of dataregarding a second color of light that exits the set of optical cavitiesat a second time.

Likewise, in some implementations: (a) ultrafast ellipsometrymeasurements are taken with an imaging system that includes a lens, aToF camera, an optical cavity, a linear polarizer that is behind thecavity, and a half-wave plate that is inside the cavity; (b) light fromthe scene enters the optical cavity; (c) different portions of light(corresponding to different polarizations of the initial light from thescene) reach the ToF camera at different times; and (d) the ToF cameramay employ any one or more of these four factors (time-of-flight,time-of-arrival, depth, and scene-roundtrip-distance) to selectivelyacquire a first set of data regarding a first portion of light(corresponding to a first polarization of the initial light from thescene) and to selectively acquire a second set of data regarding asecond portion of light (corresponding to a second polarization of theinitial light from the scene).

Likewise, in some implementations: (a) an imaging system performsultrafast filtering of spatial frequencies of light; (b) light from thescene passes through a collimating lens; (c) the collimating lensoutputs collimated light that, while still collimated, passes through aFourier plane and is incident on an unstable optical cavity; (d)different portions of light (each with a different spectrum of spatialfrequencies) reach a ToF camera at different times; and (e) the ToFcamera may employ any one or more of these four factors (time-of-flight,time-of-arrival, depth, and scene-roundtrip-distance) to selectivelyacquire a first set of data regarding light that has a first spectrum ofspatial frequencies and to selectively acquire a second set of dataregarding light that has a second spectrum of spatial frequencies.

As noted above, in illustrative implementations, any type of ToF cameramay be employed to measure, for each respective pixel of the camera: (a)one or more of these four factors (time-of-flight of incident light,time-of-arrival of incident light, scene depth, andscene-roundtrip-distance); and (b) amplitude of incident light.

In illustrative implementations of this invention, time or depthinformation (e.g., time-of-flight, time-of-arrival, depth, orscene-roundtrip-distance) may be obtained through direct or indirectmeasurement of time of arrival of the light or phase or frequency of thereceived electromagnetic wave.

For example, in some implementations of this invention, the ToF camera(e.g., 125, 425, 725, 825, 1025, 1225, or 1825) comprises a SPAD sensor(single photon avalanche diode arrays). The SPAD sensor may be operatedin TCSPC (Time correlated single photon counting mode). For example, insome implementations: (a) the SPAD sensor extracts information regardingtime-of-flight of incident light that has exited an optical cavity; and(b) a different number of roundtrips of light in an optical cavitybefore light exits the cavity results in a different time-of-flight. Forexample, when using a SPAD sensor, the measured time-of-flight (ormeasured time-of-arrival) may be different for light that exits anoptical cavity after one roundtrip in the cavity than for light thatexits the cavity after two roundtrips in the cavity. Optionally, atime-of-flight (or time-of-arrival) measured by the SPAD may be mappedto depth or to scene-roundtrip distance.

Or, for example, in some implementations of this invention, the ToFcamera (e.g., 125, 425, 725, 825, 1025, 1225, or 1825) comprises acontinuous wave time-of-flight (CW ToF) sensor. For example, the CW ToFsensor may comprise a Swiss Ranger 4000 by MESA Imaging, or a PhotonicMixer Device sensor by PMD Technologies, or a Kinect® sensor for XboxOne® by Microsoft®. In some implementations of this invention: (a) theToF camera comprises a CW ToF sensor; (b) the scene is illuminated withamplitude modulated light that is not pulsed (e.g., amplitude modulatedcontinuous wave laser light that is not pulsed); (c) the CW ToF sensorcalculates correlation of incident light with an internal electronicreference signal to extract the phase of the returnedamplitude-modulated light and then maps the phase to a depth (orscene-roundtrip-distance, time-of-flight, or time-of-arrival); and (d)each round trip of light that occurs in an optical cavity before lightexits the cavity results in a different phase that is detected by the CWToF sensor and is mapped to a different depth (or to a differentscene-roundtrip-distance, time-of-flight, or time-of-arrival). Forexample, the detected phase may be different for light that exits anoptical cavity after one roundtrip in the cavity than for light thatexits the cavity after two roundtrips in the cavity.

Or, for example, in some implementations of this invention, the ToFcamera (e.g., 125, 425, 725, 825, 1025, 1225, or 1825) comprises aso-called frequency domain CW ToF. For example:(a) the ToF camera maycomprise frequency domain CW ToF sensor; (b) the scene may beilluminated with frequency chirped CW laser light (e.g. with a chirpthat has MHz bandwidth); (c) a short-time fourier transform (STFT) ofthe reflected signal may be used to calculate a spectrogram of thereflected signal; (d) by comparison to reference chirped signal, depthinformation or time-of-flight information may be extracted; and (e) eachround trip of light that occurs in an optical cavity before light exitsthe cavity may result in a different frequency that is detected by theCW ToF sensor and is mapped to a different depth (or to a differentscene-roundtrip-distance, or to a different time-of-flight, or to adifferent time-of-arrival). For example, the detected frequency may bedifferent for light that exits an optical cavity after one roundtrip inthe cavity than for light that exits the cavity after two roundtrips inthe cavity.

In illustrative implementations, it is desirable that the ToF camera isconfigured to resolve light that exits an optical cavity at the end ofdifferent round trips of light inside the cavity (e.g., is configured todistinguish between light that exits the cavity after x roundtrips inthe cavity from light that exits the cavity after x+1 roundtrips in thecavity, were x is an integer that is greater than or equal to zero).

In some implementations of this invention: (a) the ToF camera comprisesa streak camera that captures only 1D spatial information in a singleacquisition; and (b) a periscope mirror set is employed to scan thescene vertically. In some other implementations: (a) the ToF cameracomprises a 2D ToF camera (e.g. SPAD camera, CW phase ToF camera, or CWfrequency ToF camera); and (b) a periscope mirror set is not employed.

In some implementations, folding of an optical path inside an opticalcavity may result in a measured depth (or measuredscene-roundtrip-distance) that is greater than would occur in theabsence of the folding. Likewise, in some implementations, folding of anoptical path inside an optical cavity may result in: (a) a measuredtime-of-flight that is greater than would occur in the absence of thefolding; or (b) a measured time-of-arrival that is later than wouldoccur in the absence of the folding.

In some implementations of this invention, selective acquisition of datamay occur as follows: (a) a ToF camera may acquire a large set of datawhich represents light measurements taken by a photodetector or lightsensor of the ToF camera and which is associated with many differentdepths (or scene-roundtrip-distances, or times-of-flight, ortimes-of-arrival) and thus is associated with many different portions oflight, each of which exited an optical cavity after a different numberof roundtrips of light in the cavity; (b) a subset of data, out of thislarge set of data, may be associated with a particular range oftimes-of-flight (or times-of-arrival, or depths, orscene-roundtrip-distances) and thus may be associated with light thatexited the cavity after a particular number of roundtrips; (c) thissubset of data may be selected out of the larger set of data that theToF camera captured; and (d) an image may be generated, based on thissubset of data and not on any other data. Or, for example, the selectiveacquisition may occur as follows: (a) a ToF camera may acquire a set ofdata which represents light measurements taken by a photodetector orlight sensor of the ToF camera and which is associated with only aparticular range of times-of-flight (or times-of-arrival, or depths, orscene-roundtrip-distances); and (b) an image may be generated, based onthis set of data and not on any other data.

In many implementations of this invention, the optical elements of a ToFoptical system are in a fixed spatial configuration—and have a fixedshape—throughout normal operation of the system.

Alternatively, in some implementations: (a) the relative spatialposition or shape of one or more optical elements in the imaging systemmay be adjusted during normal operation of the system (e.g., aftercapturing a first set of images and before capturing a second set ofimages); and (b) these adjustments may include changing the cavity sizeof an optical cavity (e.g., by moving a set of STMs closer to or furtherfrom each other) or changing the changing the shape or position of anoptical element (such as a lens, spatial light attenuator, mask, STM,reflective optical element, or transmissive optical element). Forexample, conventional liquid lens technology may be employed to controlthe shape of a liquid lens (e.g., by adjusting an electrical voltage).Or, for example, one or more actuators may actuate a physicaltranslation or rotation of an optical element (such as a lens or a STM).Each of these actuators may comprise any kind of actuator, including alinear, rotary, electrical, piezoelectric, electro-active polymer,mechanical or electro-mechanical actuator or MEMS actuator. In somecases, one or more sensors are configured to detect position ordisplacement and to provide feedback to one of more of the actuators.

Computers

In illustrative implementations of this invention, one or more computers(e.g., servers, network hosts, client computers, integrated circuits,microcontrollers, controllers, field-programmable-gate arrays, personalcomputers, digital computers, driver circuits, or analog computers) areprogrammed or specially adapted to perform one or more of the followingtasks: (1) to control the operation of, or interface with, hardwarecomponents of a ToF imaging system, including any light source and anyToF camera; (2) to control modulation of amplitude-modulated lightemitted by a light source; (3) to control timing of image acquisition bya camera; (4) to control timing of pulses of light emitted by a lightsource; (5) to process or post-process data acquired by a ToF camera;(6) to receive data from, control, or interface with one or moresensors; (7) to perform any other calculation, computation, program,algorithm, or computer function described or implied above; (8) toreceive signals indicative of human input; (9) to output signals forcontrolling transducers for outputting information in human perceivableformat; (10) to process data, to perform computations, to execute anyalgorithm or software, and (11) to control the read or write of data toand from memory devices (items 1-11 of this sentence referred to hereinas the “Computer Tasks”). The one or more computers (e.g. 450, 480, 481)may be in any position or positions, including housed in or external toa ToF camera or light source. The one or more computers may communicatewith each other or with other devices either: (a) wirelessly, (b) bywired connection, (c) by fiber-optic link, or (d) by a combination ofwired, wireless or fiber optic links.

In exemplary implementations, one or more computers are programmed toperform any and all calculations, computations, programs, algorithms,computer functions and computer tasks described or implied above. Forexample, in some cases: (a) a machine-accessible medium has instructionsencoded thereon that specify steps in a software program; and (b) thecomputer accesses the instructions encoded on the machine-accessiblemedium, in order to determine steps to execute in the program. Inexemplary implementations, the machine-accessible medium may comprise atangible non-transitory medium. In some cases, the machine-accessiblemedium comprises (a) a memory unit or (b) an auxiliary memory storagedevice. For example, in some cases, a control unit in a computer fetchesthe instructions from memory.

In illustrative implementations, one or more computers execute programsaccording to instructions encoded in one or more tangible,non-transitory, computer-readable media. For example, in some cases,these instructions comprise instructions for a computer to perform anycalculation, computation, program, algorithm, or computer functiondescribed or implied above. For example, in some cases, instructionsencoded in a tangible, non-transitory, computer-accessible mediumcomprise instructions for a computer to perform the Computer Tasks.

Network Communication

In illustrative implementations of this invention, electronic devices(e.g., 101, 125, 401, 425, 450, 480, 481, 725, 825, 1025, 1225) areconfigured for wireless or wired communication with other devices in anetwork.

For example, in some cases, one of more of these electronic devices(e.g., 101, 125, 401, 425, 450, 480, 481, 725, 825, 1025, 1225) may eachinclude a wireless module for wireless communication with other devicesin a network. Each wireless module (e.g., 470, 471, 472) may include (a)one or more antennas, (b) one or more wireless transceivers,transmitters or receivers, and (c) signal processing circuitry. Eachwireless module may receive and transmit data in accordance with one ormore wireless standards.

In some cases, one or more of the following hardware components are usedfor network communication: a computer bus, a computer port, networkconnection, network interface device, host adapter, wireless module,wireless card, signal processor, modem, router, cables or wiring.

In some cases, one or more computers (e.g., 450, 480, 481) areprogrammed for communication over a network. For example, in some cases,one or more computers are programmed for network communication: (a) inaccordance with the Internet Protocol Suite, or (b) in accordance withany other industry standard for communication, including any USBstandard, ethernet standard (e.g., IEEE 802.3), token ring standard(e.g., IEEE 802.5), wireless standard (including IEEE 802.11 (wi-fi),IEEE 802.15 (bluetooth/zigbee), IEEE 802.16, IEEE 802.20 and includingany mobile phone standard, including GSM (global system for mobilecommunications), UMTS (universal mobile telecommunication system), CDMA(code division multiple access, including IS-95, IS-2000, and WCDMA), orLTS (long term evolution)), or other IEEE communication standard.

Definitions

The terms “a” and “an”, when modifying a noun, do not imply that onlyone of the noun exists. For example, a statement that “an apple ishanging from a branch”: (i) does not imply that only one apple ishanging from the branch; (ii) is true if one apple is hanging from thebranch; and (iii) is true if multiple apples are hanging from thebranch.

To compute “based on” specified data means to perform a computation thattakes the specified data as an input.

Here are some non-limiting examples of a “camera”: (a) a ToF camera; (b)a streak camera; (c) a digital camera; (d) a digital grayscale camera;(e) a digital color camera; (f) a video camera; (g) a light sensor orimage sensor, and (h) a depth camera. A camera includes any computers orcircuits that process data captured by the camera.

To say that a camera “captures” an image means that the camera measuresincident light that forms the image.

Unless the context clearly indicates otherwise, “cavity” means opticalcavity.

“Common logarithm” means logarithm with base 10.

The term “comprise” (and grammatical variations thereof) shall beconstrued as if followed by “without limitation”. If A comprises B, thenA includes B and may include other things.

The term “computer” includes any computational device that performslogical and arithmetic operations. For example, in some cases, a“computer” comprises an electronic computational device, such as anintegrated circuit, a microprocessor, a mobile computing device, alaptop computer, a tablet computer, a personal computer, or a mainframecomputer. In some cases, a “computer” comprises: (a) a centralprocessing unit, (b) an ALU (arithmetic logic unit), (c) a memory unit,and (d) a control unit that controls actions of other components of thecomputer so that encoded steps of a program are executed in a sequence.In some cases, a “computer” also includes peripheral units including anauxiliary memory storage device (e.g., a disk drive or flash memory), orincludes signal processing circuitry. However, a human is not a“computer”, as that term is used herein.

“Defined Term” means a term or phrase that is set forth in quotationmarks in this Definitions section.

Unless the context clearly indicates otherwise, “depth” means distancefrom a camera to a point in a scene, when the camera is oriented suchthat the optical axis of the camera intersects the point.

The “depth resolution” of a camera means the minimum distance, betweentwo depths, such that the camera is configured to take measurements thatdistinguish between the two depths.

For an event to occur “during” a time period, it is not necessary thatthe event occur throughout the entire time period. For example, an eventthat occurs during only a portion of a given time period occurs “during”the given time period.

The term “e.g.” means for example.

The fact that an “example” or multiple examples of something are givendoes not imply that they are the only instances of that thing. Anexample (or a group of examples) is merely a non-exhaustive andnon-limiting illustration.

Unless the context clearly indicates otherwise: (1) a phrase thatincludes “a first” thing and “a second” thing does not imply an order ofthe two things (or that there are only two of the things); and (2) sucha phrase is simply a way of identifying the two things, respectively, sothat they each may be referred to later with specificity (e.g., byreferring to “the first” thing and “the second” thing later). Forexample, unless the context clearly indicates otherwise, if an equationhas a first term and a second term, then the equation may (or may not)have more than two terms, and the first term may occur before or afterthe second term in the equation. A phrase that includes a “third” thing,a “fourth” thing and so on shall be construed in like manner.

However, the preceding paragraph does not apply to numerical adjectives(e.g., first, second, third, fourth or fifth) that modify the terms“pass” and “roundtrip”. For example, in the phrases “first roundtrip”,“second roundtrip”, “third roundtrip”, “fourth roundtrip”, “fifthroundtrip”, “first roundtrip light”, “second roundtrip light”, “thirdroundtrip light”, “fourth roundtrip light”, and “fifth roundtrip light”,the numerical adjectives are ordinal terms that indicate a temporalorder.

To say that light travels in a “first pass” in an optical cavity meansthat light travels in a path that: (a) begins when the light enters thecavity; (b) ends when the light exits the cavity; and (c) does notinclude a roundtrip in the cavity.

“FP” means Fabry-Perot.

As used herein, a “Fourier plane” does not need to be an ideal Fourieroptical plane. A non-limiting example of a “Fourier plane” is ageometric plane at which an actual light signal is a sufficiently closeapproximation (to that which would occur at a Fourier optical planeunder ideal Fourier plane conditions) that a person skilled in the artwould consider the geometric plane to function, for practical purposes,as a Fourier optical plane.

Light “from” a scene means light that has traveled, directly orindirectly, from the scene. Unless the context clearly indicatesotherwise: (a) each description herein of light “from” the scene doing X(e.g., passing through an optical cavity and lens) is meant to describethe light doing X at a time after the light left the scene; and (b) eachdescription herein of light “from” the scene having X done to it (e.g.,being collimated) is meant to describe X being done to the light at atime after the light left the scene.

In the context of an imaging system that captures an image of a scene,to say that B is located in “front” of C means that: (1) an optical pathpasses through B and C; and (2) the optical distance between B and thescene is less than the optical distance between C and the scene. In thecontext of an imaging system that captures an image of a scene, to saythat B is located “behind ” C means that: (1) an optical path passesthrough C and B; and (2) the optical distance between B and the scene ismore than the optical distance between C and the scene.

To say a “given” X is simply a way of identifying the X, such that the Xmay be referred to later with specificity. To say a “given” X does notcreate any implication regarding X. For example, to say a “given” X doesnot create any implication that X is a gift, assumption, or known fact.

“Herein” means in this document, including text, specification, claims,abstract, and drawings.

To “image” means to capture an image.

As used herein: (1) “implementation” means an implementation of thisinvention; (2) “embodiment” means an embodiment of this invention; (3)“case” means an implementation of this invention; and (4) “use scenario”means a use scenario of this invention.

The term “include” (and grammatical variations thereof) shall beconstrued as if followed by “without limitation”.

“Intensity” means any measure of intensity, energy or power. Forexample, the “intensity” of light includes any of the followingmeasures: irradiance, spectral irradiance, radiant energy, radiant flux,spectral power, radiant intensity, spectral intensity, radiance,spectral radiance, radiant exitance, radiant emittance, spectral radiantexitance, spectral radiant emittance, radiosity, radiant exposure,radiant energy density, luminance or luminous intensity.

“Lens” means a single lens, compound lens, or lens system.

“Light” means electromagnetic radiation of any frequency. For example,“light” includes, among other things, visible light and infrared light.Likewise, any term that directly or indirectly relates to light (e.g.,“imaging”) shall be construed broadly as applying to electromagneticradiation of any frequency.

To say that A is “located at” at B means that the spatial position of Ais at B.

“LP” means linear polarizer.

To say that a filter reflects a “majority” of light in a subband offrequencies that is incident on the filter means that, for the subbandtaken as a whole, the radiant flux of light that is reflected from thefilter is greater than 50% of the radiant flux of light that is incidenton the filter. To say that a filter transmits a “minority” of light in asubband of frequencies that is incident on the filter means that, forthe subband taken as a whole, the radiant flux of light that istransmitted through the filter is less than 50% of the radiant flux oflight that is incident on the filter. In the preceding two sentences,the only light that is counted is light that is in the subband.

“ND filter” means a neutral density filter.

“Nominal focal length” is defined above.

An “optical cavity” or “OC” means an optical resonator which isconfigured such that light reflecting inside the resonator tends, overtime, to form standing waves for resonance frequencies. Hardware thatsatisfies the definition in the preceding sentence is an “opticalcavity” even at a time when there is no light reflecting inside thehardware and even during initial roundtrips of light when resonanceeffects are not yet significant. An “optical cavity” may be eitherstable or unstable.

The “optical density” (or “O.D.”) of a surface means the commonlogarithm of the ratio of the radiant flux incident on the surface tothe radiant flux transmitted through the surface.

An “optical path” is a path that light travels. As a non-limitingexample, all or part of an “optical path” may be through air, a vacuumor a lens. As a non-limiting example, if an imaging system is configuredsuch that at least a portion of light that enters the optical systemtravels along a given path, then the given path is an “optical path” inthat imaging system, even at a time when light is not traveling on thatpath.

The term “or” is inclusive, not exclusive. For example, A or B is trueif A is true, or B is true, or both A or B are true. Also, for example,a calculation of A or B means a calculation of A, or a calculation of B,or a calculation of A and B.

Unless the context clearly indicates otherwise, “path” means an opticalpath.

“Polarizer” means an optical filter which (a) allows light that has aspecific polarization (or is in a specific range of polarizations) topass, and (b) blocks light with other polarizations.

A “ring optical cavity” or “ring cavity” means an optical cavity whichis configured such that light strikes a given optical element only onceduring a roundtrip of light in the cavity.

To say that light travels in a “roundtrip” in an optical cavity meansthat light travels in a path that begins and ends at the same opticalelement (e.g., at the same STM) of the cavity. However, a “roundtrip”does not necessarily begin and end at the exact same spatial point. Forexample, in an unstable optical cavity, a roundtrip may start at a firstspatial point on an STM and may end at a different spatial point on thesame STM.

“Roundtrip length” means the total distance that light travels in asingle, complete roundtrip inside an optical cavity.

As used herein, the term “set” does not include a group with noelements. Mentioning a first set and a second set does not, in and ofitself, create any implication regarding whether or not the first andsecond sets overlap (that is, intersect).

A “semi-transparent mirror” or “STM” means an optical element that ispartially transmissive and partially reflective, such that when lightstrikes the STM, a portion of the light is reflected by the STM and aportion of the light is transmitted through the STM.

Unless the context clearly indicates otherwise, “some” means one ormore.

A “spatial light attenuator”, also called an “SLA”, means a device that(i) either transmits light through the device or reflects light from thedevice, and (ii) attenuates the light, such that the amount ofattenuation of the light depends the spatial position at which the lightis incident on the device. Non-limiting examples of an SLA include amask, an LCD (liquid-crystal display) and a DMD (digital micromirrordevice).

“Substantially” means at least ten percent. For example: (a) 112 issubstantially larger than 100; and (b) 108 is not substantially largerthan 100.

The term “such as” means for example.

To say that A happens and “then” B happens, means that A happens beforeB happens.

A statement that light passes through A “and” B: (a) is true if lightpasses through A and then passes through B; and (b) is true if lightpasses through B and then passes through A. For example, a statementthat light passes through a cavity “and” a lens: (a) is true if lightpasses through the cavity and then passes through the lens; and (b) istrue if light passes through the lens and then passes through thecavity.

To say that light which passes through an optical cavity is filtered bya filter does not imply a temporal order in which the passing andfiltering occur. For example, a statement that “light that passesthrough an optical cavity is filtered by a spectral filter”: (a) is trueif the light is filtered by the filter and then passes through thecavity; and (b) is true if the light passes through the cavity and thenis filtered by the filter.

The “temporal resolution” of a camera means the minimum time difference,between a first time-of-arrival of light and a second time-of-arrival oflight, such that the camera is configured to take measurements thatdistinguish between the two times-of-arrival.

“Time-of-arrival” means the time at which light arrives at a camera.

In the context of light that travels from a camera to a scene and thenback to the camera, “time-of-flight” means the amount of time that ittakes for the light to travel from the camera to the scene and then backto the camera.

“Time-of-flight camera” or “ToF camera” means a camera that has atemporal resolution which is less than 3 nanoseconds or that has a depthresolution which is less than 1 meter.

To say that a machine-readable medium is “transitory” means that themedium is a transitory signal, such as an electromagnetic wave.

“Unstable” is defined above.

A matrix may be indicated by a bold capital letter (e.g., D). A vectormay be indicated by a bold lower-case letter (e.g., a). However, theabsence of these indicators does not indicate that something is not amatrix or not a vector.

An “x-t streak image” means an image that: (a) is captured by a streakcamera; (b) measures spatial information in only one spatial dimensionx; and (c) has two axes, x and t, where the t axis represents time andthe x axis represents the spatial dimension x.

Except to the extent that the context clearly requires otherwise, ifsteps in a method are described herein, then the method includesvariations in which: (1) steps in the method occur in any order orsequence, including any order or sequence different than that described;(2) any step or steps in the method occurs more than once; (3) any twosteps occur the same number of times or a different number of timesduring the method; (4) any combination of steps in the method is done inparallel or serially; (5) any step in the method is performediteratively; (6) a given step in the method is applied to the same thingeach time that the given step occurs or is applied to different thingseach time that the given step occurs; (7) one or more steps occursimultaneously, or (8) the method includes other steps, in addition tothe steps described herein.

Headings are included herein merely to facilitate a reader's navigationof this document. A heading for a section does not affect the meaning orscope of that section.

This Definitions section shall, in all cases, control over and overrideany other definition of the Defined Terms. The Applicant or Applicantsare acting as his, her, its or their own lexicographer with respect tothe Defined Terms. For example, the definitions of Defined Terms setforth in this Definitions section override common usage or any externaldictionary. If a given term is explicitly or implicitly defined in thisdocument, then that definition shall be controlling, and shall overrideany definition of the given term arising from any source (e.g., adictionary or common usage) that is external to this document. If thisdocument provides clarification regarding the meaning of a particularterm, then that clarification shall, to the extent applicable, overrideany definition of the given term arising from any source (e.g., adictionary or common usage) that is external to this document. To theextent that any term or phrase is defined or clarified herein, suchdefinition or clarification applies to any grammatical variation of suchterm or phrase, taking into account the difference in grammatical form.For example, the grammatical variations include noun, verb, participle,adjective, and possessive forms, and different declensions, anddifferent tenses.

Variations

This invention may be implemented in many different ways. Here are somenon-limiting examples:

In some implementations, this invention is a system comprising a lens,an optical cavity and a time-of-flight (ToF) camera, wherein the systemis configured to capture an image of a scene, such that the image isformed by light that is from the scene and that passes through theoptical cavity and the lens. In some cases, the lens is located in frontof the optical cavity. In some cases: (a) a rear focal plane of the lensis located at a distance from the lens; and (b) the distance is lessthan the nominal focal length of the lens, due to folding of opticalpaths in the optical cavity. In some cases, all or part of the lens islocated inside the optical cavity. In some cases: (a) the system isconfigured to capture a first focused image at a first depth in a sceneand a second focused image at a second depth in the scene, such that, atall times during the capture of the first and second images, all opticalelements of the system remain in a fixed shape and remain stationaryrelative to each other and to the scene; and (b) the first depth is lessthan half of the second depth. In some cases, the lens is located behindthe optical cavity and in front of the ToF camera. In some cases: (a)the system includes a first filter that is a neutral-density filterwhich is configured to allow a band of frequencies of light to passthrough the first filter; (b) the band of frequencies includes a firstsubband of frequencies and a second subband of frequencies, the firstband being different than the second band; (c) the system also includesa second filter and a third filter; (d) the second filter comprises aspectral filter that is configured (i) to reflect a majority of, and totransmit a minority of, light in the first subband of frequencies and(ii) to transmit a majority of light in the second subband offrequencies; (e) the third filter comprises a spectral filter that isconfigured (i) to reflect a majority of, and to transmit a minority of,light in the second subband of frequencies and (ii) to transmit amajority of light in the first subband of frequencies; (f) the firstfilter is in front of the second filter and the second filter is infront of the third filter; (g) the first and second filters comprise theoptical cavity mentioned in the first sentence of this paragraph (firstoptical cavity); and (h) the first and third filters comprise a secondoptical cavity. In some cases: (a) the ToF camera has a temporalresolution; and (b) the system is configured to have an impulse responsesuch that, in response to an input that consists of a pulse of lightfrom the scene entering the first filter (i) a first set of exit pulsesexits the first optical cavity and a second set of exit pulses exits thesecond optical cavity, (ii) the first set of exit pulses includes afirst pulse which is the earliest pulse that exits the first opticalcavity after the input, (iii) the second set of exit pulses includes asecond pulse which is the earliest pulse that exits the second opticalcavity after the input, and (iv) the first pulse arrives at the ToFcamera at a first time and the second pulse arrives at the ToF camera ata second time, such that the absolute value of the first time minus thesecond time is greater than the temporal resolution of the ToF camera.In some cases: (a) the system further comprises a waveplate and apolarizer; (b) all or part of the waveplate is located inside theoptical cavity; and (c) the polarizer is behind the optical cavity andin front of the ToF camera. In some cases, the system is configured suchthat light passes through the waveplate during each roundtrip of lightin the optical cavity. In some cases: (a) the optical cavity isunstable; (b) the system further comprises a collimating lens and aspatial light attenuator; (c) the collimating lens is configured tooutput collimated light that, while collimated (i) passes through aFourier optical plane and (ii) then is incident on the optical cavity;(d) all or part of the spatial light attenuator is located inside theoptical cavity; (e) the collimating lens is in front of the opticalcavity; and (f) the lens mentioned in the first sentence of thisparagraph is behind the optical cavity. In some cases, the system isconfigured such that the image is formed by light that exited theoptical cavity before completing more than one hundred roundtrips in theoptical cavity. In some cases, the system is configured such that: (a)the optical cavity has a dominant mode; (b) a wavefront of light evolvesover time inside the optical cavity; and (c) the image is formed bylight that exits the optical cavity at a time when the integral power ofthe dominant mode of the optical cavity is less than the half-maximum ofthe integral power of the wavefront inside the optical cavity. In somecases, the light is pulsed. In some cases, the light is not pulsed. Insome cases: (a) the system further comprises a second optical cavity, afirst spectral filter, and a second spectral filter; (b) the firstspectral filter is configured to allow only a first frequency band oflight to pass through the first spectral filter and the second spectralfilter is configured to allow only a second frequency band of light topass through the second spectral filter, the first band being differentthan the second band; (c) the second spectral filter is positioned,relative to the second optical cavity, such that light that passesthrough the second optical cavity is filtered by the second spectralfilter; (d) the first spectral filter is positioned, relative to theoptical cavity mentioned in the first sentence of this paragraph (firstoptical cavity), such that light that passes through the first opticalcavity is filtered by the first spectral filter; and (e) a roundtriplength of the first optical cavity is substantially different than aroundtrip length of the second optical cavity. In some cases: (a) thetime-of-flight camera has a temporal resolution; and (b) the system isconfigured to have an impulse response such that, in response to aninput that consists of a pulse of light entering the first and secondspectral filters simultaneously (i) a first set of exit pulses exits thefirst optical cavity and a second set of exit pulses exits the secondoptical cavity, (ii) the first set of exit pulses includes a first pulsewhich is the earliest pulse that exits the first optical cavity afterthe input, (iii) the second set of exit pulses includes a second pulsewhich is the earliest pulse that exits the second optical cavity afterthe input, and (iv) the first pulse exits the first optical cavity at afirst time and the second pulse exits the second optical cavity at asecond time, such that the absolute value of the first time minus thesecond time is greater than the temporal resolution of thetime-of-flight camera. Each of the cases described above in thisparagraph is an example of the system described in the first sentence ofthis paragraph, and is also an example of an embodiment of thisinvention that may be combined with other embodiments of this invention.

In some implementations, this invention is a method comprising capturingan image of a scene, such that the image is captured by a time-of-flight(ToF) camera and is formed by light that (i) is from the scene and (ii)passes through an optical cavity and a lens before reaching the ToFcamera. In some cases, the light passes through the lens before passingthrough the optical cavity. In some cases, all or part of the lens islocated inside the optical cavity. In some cases, wherein the lightpasses through the optical cavity before passing through the lens. Insome cases, the method further comprises: (a) passing the light througha waveplate, while the light is inside the optical cavity; and (b)passing the light through a linear polarizer, after the light exits theoptical cavity. In some cases: (a) the optical cavity is unstable; and(b) the method further comprises passing the light from the scenethrough an additional lens that outputs collimated light which, afterexiting the additional lens and while still collimated (i) passesthrough a Fourier optical plane and (ii) then is incident on the opticalcavity. Each of the cases described above in this paragraph is anexample of the method described in the first sentence of this paragraph,and is also an example of an embodiment of this invention that may becombined with other embodiments of this invention.

Each description above of any method or apparatus of this inventiondescribes a non-limiting example of this invention. This invention isnot limited to those examples, and may be implemented in other ways.

Each description above of any implementation, embodiment or case of thisinvention (or any use scenario for this invention) describes anon-limiting example of this invention. This invention is not limited tothose examples, and may be implemented in other ways.

Each Figure that illustrates any feature of this invention shows anon-limiting example of this invention. This invention is not limited tothose examples, and may be implemented in other ways.

The Provisional Application does not limit the scope of this invention.The Provisional Application describes non-limiting examples of thisinvention, which examples are in addition to—and not in limitationof—the implementations of this invention that are described in the mainpart of this document. For example, if any feature described in theProvisional Application is different from, or in addition to, thefeatures described in the main part of this document, this additional ordifferent feature of the Provisional Application does not limit anyimplementation of this invention described in the main part of thisdocument, but instead merely describes another example of thisinvention. As used herein, the “main part of this document” means thisentire document (including any drawings listed in the Brief Descriptionof Drawings above), except that the “main part of this document” doesnot include any document that is incorporated by reference herein.

The above description (including without limitation any attacheddrawings and figures) describes illustrative implementations of theinvention. However, the invention may be implemented in other ways. Themethods and apparatus which are described herein are merely illustrativeapplications of the principles of the invention. Other arrangements,methods, modifications, and substitutions by one of ordinary skill inthe art are therefore also within the scope of the present invention.Numerous modifications may be made by those skilled in the art withoutdeparting from the scope of the invention. Also, this invention includeswithout limitation each combination and permutation of one or more ofthe implementations (including hardware, hardware components, methods,processes, steps, software, algorithms, features, or technology) thatare described or incorporated by reference herein.

What is claimed is:
 1. A system comprising a lens, an optical cavity anda time-of-flight camera, wherein: (a) the system is configured tocapture an image of a scene, in such a way that the image is formed bylight that is from the scene and that passes through the optical cavityand the lens; (b) the lens is located in front of the optical cavity;(c) a rear focal plane of the lens is located at a distance from thelens; and (d) the distance is less than the nominal focal length of thelens, due to folding of optical paths in the optical cavity.
 2. A systemcomprising a lens, an optical cavity and a time-of-flight camera,wherein: (a) the system is configured to capture an image of a scene, insuch a way that the image is formed by light that is from the scene andthat passes through the optical cavity and the lens; (b) all or part ofthe lens is located inside the optical cavity; (c) the system isconfigured to capture a first focused image at a first depth in a sceneand a second focused image at a second depth in the scene, in such a waythat, at all times during the capture of the first and second images,all optical elements of the system remain in a fixed shape and remainstationary relative to each other and to the scene; and (d) the firstdepth is less than half of the second depth.
 3. A system comprising alens, an optical cavity and a time-of-flight camera, wherein: (a) thesystem is configured to capture an image of a scene, in such a way thatthe image is formed by light that is from the scene and that passesthrough the optical cavity and the lens; and (b) the system isconfigured in such a way that the image is formed by light that exitedthe optical cavity before completing more than one hundred roundtrips inthe optical cavity.